Log Integrity Overview¶
Almost every system needs to keep audit logs – information on “who did what and when”. For example:
- Who changed the patient’s medical history
- Which bank employee viewed the customer bank account
- Which CCTV camera has taken a particular video and when
It’s not just important who did it, but also what exactly they changed or viewed and when. The whole set of details (actor, action, entity modified, details of the action) has to be stored. But simply storing it won’t be sufficient, as the stored data can be modified – there’s a need for securely storing it and guaranteeing its integrity.
Whenever confronted with the need to store an audit log (which is the case for almost all applications), companies go for custom home-grown solutions. In the best case scenario they use some plug-in to their database access technology (e.g. an ORM) that automatically handles modifications. Very rarely these solutions cover the following set of requirements:
- Business-logic level events, network events as well as insert/update/delete/get events have to be stored – it’s not sufficient to just see the database rows that were updated, sometimes high-level operations like “basket checkout”, “bank transfer initiated” or “medical examination performed” need to be stored as well, as they carry more meaning to the business than a bunch of database table updates.
- The audit-log has to be tamper-proof, i.e. nobody, even system administrators, can alter it without being detected. That way the audit log has a more significant legal strength in courts. And the business can make certain guarantees to their clients.
- The audit-log has to be easily searchable and navigable – anyone with proper access (a high level manager, line manager or even external auditor) can easily see sequences of events that lead to a particular issue.
And when speaking of issues – data manipulation attacks, both from insiders and outside attackers, are a serious threat nowadays. Wired has put it in the top security threats prediction for 2016 . Without taking additional measures, no business is safe from having their data manipulated for the benefit of other parties. And often without even detecting that until it’s too late (this is why NIST recommends as well as OWASP recommend using audit logs).
Last, but not least, public sector entities, as well as regulated businesses, need to comply with a set of security regulations. Ticking the “compliance” box may not be that hard, but the liabilities for not protecting customers’/citizens’ data are still a serious risk to be taken into account.
Speaking of regulations, the General Data Protection Regulation (GDPR) in the EU mandates that every system that processes personal data (which is true for most systems) must have an audit log and not complying with the regulation may lead to significant fines for any business.
While many companies think they have audit logs, they are almost never properly protected. And as one book on ISO 27001 warns:
System logs need to be protected, because if the data can be modified or data in them deleted, their existence may create a false sense of security. There needs to be a product that addresses all of the above concerns and observations, and LogSentinel aims to be such a product.
While the integrity of the data itself is not the immediate goal of the LogSentinel SIEM (as opposed to protecting the integrity of the audit log), having a full secure audit log on all data changes means that the audited data is also protected – you can always trace the data modifications in the log and verify whether a given change occurred “naturally”, or was altered inappropriately.
The technology used is based on a number of peer-reviewed papers, e.g. Audit Logs to Support Computer Forensics by Bruce Schneier and John Kelsey and Efficient Record-Level Keyless Signatures for Audit Logs by Ahto Buldas et al.
The implementations is customized to account for the various business cases that should be addressed. Some properties have been dropped or moved to the client/client libraries, and the need for scaling the product and providing multi-tenancy has lead to a bit more complicated processing logic.
Notably, the technology relies on consecutively hashing all incoming audit log events, where each subsequent hash is formed by the data of the current entry combined with the hash of the previous entry. That way the audit log entries form a hash chain that cannot be “broken” – i.e. any manipulation to any of the entries will result in invalid hashes from that moment on.
In addition to forming a hash chain, entries form groups, which are timestamped using a local timestamping authority. The group is represented by a single value, which is the root of the merkle tree of the hashes of all the entries in a group. A merkle tree is a data structure used in the blockchain to guarantee the integrity of each block. The timestamping provides additional integrity guarantees and legal strength (Regulation (EU) 910/2014 defines the rules for trusted timestamping). The client can provide their own timestamping authority which is used on regular intervals to timestamp a list of the groups (blocks) generated above.
Additionally, a constantly growing merkle tree is generated from all the entires for a particular data source (chain). The gree generation process happens every 6 hours by default and the newly obtained merkle root is stored externally (see the next paragraph).
The integrity of the log is guaranteed even if a malicious actor gains access to the log database. So you don’t have to trust the LogSentinel cloud service, or even your own employees who manage the hosted solution. In order to achieve that, there is one "catch" – the latest hash in the hash chain at a given moment has to be kept in an unmodifiable way. When you have a given hash, the fact that it is present in the audit log guarantees that it hasn’t been tampered with. And vice-versa – if a hash that was previously stored is missing from the audit log, it means the log has been tampered with. There are multiple ways to store these latest hashes in an unmodifiable way, and LogSentinel supports:
- Store it on the Ethereum blockchain. Public blockchains are the perfect candidate, as they are immutable – once an entry is stored there, it cannot be removed. Transaction fees are relatively cheap, so LogSentinel can regularly push the last known hashes to the Ethereum blockchain
- Print it on paper – it can be published in newspapers (which is less practical), or printed on a blank paper and stored in physically protected cases, or snail-mailed to multiple stakeholders, including auditors
- Store it on a write-only medium. Be it a CD-R, or more generally – any WORM storage
- Email it to multiple stakeholders – while email can be manipulated as well, having it distributed to multiple people, potentially with different email servers, increases the complexity of changing the hash in all places
On each log request, LogSentinel returns the latest known processed hash which you can decide when and how to store.
Sometimes digital evidence has to be presented in court, or to regulators. It carries additional strength if you can prove to 3rd parties that your data hasn't been tampered with. In fact, in any other scenario, the opposite side can dispute the truthfulness of logs. With LogSentinel SIEM's cryptographic integrity guarantees, and leveraging existing legal framework (e.g. the EU eIDAS regulation), the collected logs carry significant legal strength in court proceedings and otherwise.
LogSentinel SIEM serves as a secure audit log service that is simple to integrate and guarantees the integrity of all your audit data. It can be integrated in any system with minimal effort. The use of state-of-the-art cryptography and original research makes sure the data is verifiably protected and this can be proven to both customers and law enforcement.