Not all Log Management solutions are created equal... Trusting your logs.

Log Integrity is at the core of using logs for such purpose as building Trust, providing non-repudiation and indisputable proof in business relationships between Customers and Providers, but also to provide for evidence admissible in a court of law. We saw that not all Log Management solutions are created equal, and we saw some high-level requirements in terms of log collection and log reporting. We need a solution that is simple to deploy - we want an enabler, not a disabler - and a solution that allows a very rich set of APIs to accommodate for very different reporting on all kinds of metrics.

There are 2 other Critical Success Factors that need to be part of the "Not all Log Management Solutions are Created Equal" equation, and these are Log Integrity and Provider Reversibility. Today, more on Log Integrity.

Log Integrity
Logs can be used to foster Trust by providing non-repudiation and indisputable proof... provided that we can Trust the logs, in other words if we can guarantee their integrity. This is important not only for Trust for business relationship between a customer and its provider, but also in case of security breach as logs become prime evidence and need to be admissible in a court of law.  Imagine seeing malicious behavior but not being able to use your logs as evidence because you cannot guarantee that they have not been tampered with. It's like if you knew a crime was committed and you even have a picture of it, but this evidence is thrown away because you cannot prove that it was not photoshop'ed. Too bad...

We need 3 different proofs of integrity are demonstrated;

1. Proof of integrity of each log - demonstrate that no log has been altered.
2. Proof of integrity of the log sequence - demonstrate that no log has been added and no log has been deleted.
3. Proof of integrity of the report - demonstrate that the report is complete and that all logs are reported on.

Once all of these are provided, there is Explicit Trust in raw logs. There are many ways of providing Log Integrity and Log Sequence Integrity, let's have a look at one of the easiest ways, to create a digitally signed - or at least a one-way hashed - chained file of logs.

In the following diagram Figure 7, we see how this can provide for log integrity and log sequence integrity.

Figure 7: Proof of log integrity through log block chaining and signing

In the following diagram Figure 8, we see that any modification of a log or any modification of the log sequence will be immediately detected and that we’ll be able to claim loss of integrity in logs. Without getting into implementation considerations, there are obvious tradeoffs on the size of each log block. The longer the block, the easier the management and the better the performance; the shorter the block, the fewer logs we have to throw away if we were to detect loss of integrity.

Figure 8: Loss of integrity detected

We can now trust the information contained in the raw logs as being genuine, and we can trust the information contained in the reports as being non-tainted. Report completeness needs to be guaranteed by the tool, in other words, there needs to be built-in mechanisms that insure that all logs that need to be part of a report are included in the report generation and computation. This is an inherent function of the tool. We can never prevent accidental or malicious modification of a log, but we can detect modifications with a simple yet powerful way, Log Block Chaining and Signing. This will insure that the logs that we work from are genuine, have not been modified, and represent a clean source of data on top on which we can build non-repudiation and proof of claim, we can claim Trust.