« May 2005 | Main | September 2005 »

June 30, 2005

Let go!

I had a conversation recently with someone about the dangers of automating too much functionality in our information technology systems. We were talking about security and he was worried that automatically stopping attacks could cause a lot of collateral damage through false positives. For this reason, he would much rather keep a human in the loop. Furthermore, he was very wary of mechanisms that he couldn't understand or see the inner workings of. He didn't want anything that would function as a "black box".

Legitimate concerns, indeed, but shortsighted none-the-less. With the growing complexity of systems we can no longer maintain fine control over them, we need systems that look after themselves, and to do that they need to be autonomous and adaptive. And as soon as they become autonomous and adaptive, they will very likely move outside the realm of being easy to understand. But I would argue that this is a good thing, not a bad thing. We have to undestand the implications, yes, but we also have to surrender control. It's the only way forward.

A good way to illustrate this is to think of how useful horses have been to humanity throughout history. We can control them and understand very well what to expect from them, but for most of our history we had no idea whatsoever of how they work inside. In future, we will have machines and computing systems that we don't understand at all, but that function much better than they do today, that repair themselves, and adapt and evolve autonomously.

When we think of this kind of future, a key question is how do we know such systems work? Well, in the same way that we know that horses work: by observation and through black-box testing. We know how fast a typical horse can run and for how long because we have "tested" so many of them. We don't have any guarantees however - we could get a horse that was lame and so would fall far outside the expected performance. But that is a chance we take, and it's a small price to pay for a horse that fights off most infections without any assistance whatsoever, that can learn and grow and survive autonomously.

We need to let go: it's time for our systems to look after themselves so that one day they may look after us.

June 16, 2005

On the Virulence of Malware

In a previous post, I talked about the evolution of malware and speculated that we might see malware that evolves to be useful, similarly to the way mitochondria have evolved to be useful to us. So it was with great interest that I read a recent article in Security Focus about the evolution of malware, in which they argue that malware is becoming more stealthy and more benign (non-destructive) in terms of damage done to the host computer. They claim that because malware is more useful to the attacker (eg for financial gain) the malware is less likely to have a destructive payload. To support this, they present data that shows a distinct trend of the last five years in the growth of non-destructive malware. This is the key graphic in their paper:

The authors claim that this is a perfectly logical consequence of the fact that malware is becoming a "truly successful parasite". They list three conditions for a parasite to be "truly successful":

1. it spreads rapidly and effectively;
2. it does not cause a violent adverse reaction in its host such that it is rapidly destroyed;
3. it is able to extract valuable resources from its host.

This is a rather naive view of parasites, at least from a biological perspective. In the biological world, "success" is defined purely by survival and propagation: the more copies there are of a parasite, the more successful it is. We can look at this measure of success in different ways, for example, if a parasite can retain high numbers across a variety of environmental changes, then it is even more successful (robustness to environmental perturbations). The issue is that this definition of "success" introduces a new tension that may favor destructive behavior towards the host: increasing the number of copies of a parasite greatly increases the chance of it being virulent, i.e. harming the host.

In fact, the evolution of virulence of a parasite is not well understood and is very complex, depending on a variety of factors, including:

1. Transmission mode is theorized to be the main determinant of virulence, but the relation is complex; for example, virulence in malaria is postively correlated with transmission up to a certain limit, but after that excessive virulence is selected against because harm to the host will cause prevention of transmission.
2. Some results indicate that the more "durable" the pathogen (its ability to survive outside the host), the more virulent it will be because it has low reliance on host survival. This has been termed the "curse of the Pharoah" because Lord Carnarvon was thought to have died of an extremely durable pathogen he was infected with upon opening the tomb of Tutankhamun.
3. Hard selection (the death of an individual is independent of others) results in higher virulence because there is pressure to produce more copies, which causes harm. An example of this is a model demonstrating that immune responses boost virulence.
4. By contrast with the previous point, soft selection (the death of an individual is dependent on others, e.g. through competition) can actually lead to lower virulence. This is a consequence of interference between pathogens reducing host exploitation.
5. Vaccinations can have differing effects on virulence: imperfect vaccines are likely to increase virulence, whereas perfect vaccines that block transmission are likely to decrease virulence.

The above list is by no means exhaustive and there is a lot of debate about these various factors. One thing that is clear is that it is extremely hard to generalize. An article in Proc Biol Sci (2001 Nov 22;268(1483):2331-7) cogently makes this point:

Consequently, a detailed mechanistic description of how parasites and other mortality sources combine to cause host mortality is required before reliable predictions about virulence evolution can be made.

Given the complexity of the evolution of virulence in biological parasites, it is interesting to speculate on how particular factors in computer systems could influence the evolution of virulence in computer malware. As in biological systems, the picture is anything but simple.

First, we have to understand what it means to be successful for malware. This can be quite varied. For self-replicating malware such as worms and viruses, success could be very similar to that in biological systems: the number of copies extant. But if we define success in terms of the malware writers' goals, then there could be many definitions. Some goals will lead to destructive malware and some won't. Here are some ideas:

1. Attackers that want to steal information or use the resources of victims for spam relays or other such purposes are likely to favor reduced virulence, because the computers are valuable resources.
2. Attackers might alternatively want to take down machines in Denial-of-Service style attacks, or corrupt data, either as part of an extortion racket, or industrial sabotage, or terrorism or for other reasons. In this case, damage to the host is probably mandatory.
3. The increase in the variety of malware and the number of malware authors, and the increase in the number of malware on any one computer is likely to lead to increased virulence because there will be more buggy malware causing problems, and there will be more unforeseen contention between malware. We can already see evidence of this in the way spyware can clog up machines and render them virtually unuseable.
4. Some of the biological results mentioned above indicate that an immune response can increase virulence. Does this imply that security mechanisms for computers would also be likely to cause virulence?
5. What are the effects of hard and soft selection? In the biological world, there is some evidence that soft selection decreases virulence. Does this imply that having multiple varieties of malware on your computer will decrease their virulence as they compete amongst themselves? This would be counter to the argument I made in point 3 above!
6. What about malware that causes harm as a side-effect of spreading very rapidly, like the SQL Slammer worm? Clearly, if the malware authors intent is to get it on every machine for purposes such as information stealing, then a noisy, visible worm that overloads the network is a bad idea. However, what happens if a new vulnerability emerges and multiple worm writers decide to exploit it in order to get malware on as many machines as possible? If each worm closes off the vulnerability behind it, then the worm that moves the fastest will be the most successful in compromising the most machines. But that very factor increases virulence, at least in terms of network load. It is unavoidable, just as it is in the biological world. Will we see such races in future, increasing virulence?

This certainly is a complex and fascinating topic. I would love to see the writers of the Security Focus article extend their analyses to consider more factors. I will certainly be an avid reader!

June 09, 2005

CSI NetSec

If anyone reading this is going to be at CSI NetSec in Scottsdale Az next week, I'm giving a talk at 9:15 on Tuesday the 14th about ideas we can glean from biology to help secure computer systems. I'll talk a bit about immunology and how we can apply ideas from the study of immune systems to computer security. I'll talk about layers in the immune system and in security, about adaptivity in the immune system and how that is missing in our security systems. I'll also talk about the role of human intervention, both in immunity (for example, through vaccinations) and in security. I'll talk about response, how the immune system handles false positives, and how we need to redesign systems to be more secure, but not in the traditional way of having a secure or trusted codebase to build on, but a system in which no part is assumed to be secure or trusted.

So if you're there, come along!

June 02, 2005

The benefits of biomimicry

Scientific American has a fascinating article on neuromorphic computing (subscription required). The idea is to build computer systems that mimic biological systems at the level of hardware, not just software. This allows the researchers to build silicon retina chips that are remarkably effective at decoding images and do so in a way that is sufficiently compact and energy efficient that an artificial retinal implant is a possibility.

The key is that bioloigcal "computing" systems are evolved to serve particular roles, and hence are optimized for those roles (as much as evolution really optimizes anything). Biological systems cannot afford too much bloat. Inefficiency costs, and excessive functionality or computing power that is not used will rapidly get evolved out of the system. Even on the non-evolutionary level, the adaptivity of the body is such that your muscles will atrophy if you don't exercise them - the body will not waste resources on what is not being used. Parsimony is everything, and by incorporating this principle, we can build amazing new technology such as neuromorphic retinal implants.

Unfortunately, in computing systems, we have always preferred to build general purpose machines, rather than machines which fulfill a particular role. It's much easier to build such machines and saves us from the headache of trying to figure out what the machine may be used for beforehand. It also allows us more flexibility in allocating resources. All of this makes sense in a world where computing technology is rare and expensive, but as the cost of computing drops dramatically, so does the need for using general purpose hardware and software.

The problem with general purpose systems is not only that they are inefficient, but that they are much more prone to both errors and abuse, because of the excess functionality they support. Many computing systems would be a great deal more secure if the system was limited in functionality to what was actually needed. But this seldom happens - after all, it's much easier to slap a new linux kernel onto a system with all its functionality, than to take the time and effort to pare it down to the minimum necessary. In fact, constraining excessive functionality is precisely what a lot of security tools do, including many in the intrusion prevention category.

There is a dangerous trend of using general purpose computing platforms where highly specific sytems used to be deployed. For example, Microsoft recently announced its point-of-sale server based on the MS operating system. This seems like a very bad idea - in future, you will have to update your point-of-sale servers as well as all your PCs, and if a worm strikes, it could bring down everything. This is the wrong way to go: from dedicated hardware/software systems, to general purpose systems that have excessive bloat and far too much functionality/power for the task at hand. As another example, a recent article describes testing the "brainy Acura" car, which has an onboard computer that does a plethora of functions, but during the test, the computer had to be rebooted!

But I believe there is hope. Although bloated general-purpose systems are likely to be deployed in places like cars and stores, fortunately mobile devices (such as cell phones) exert a pressure against bloat. As mobile devices become more compact and more widely used, the biggest limiting factor for their usefulness will be battery power. That will drive the need for specialized hardware and software, designed to serve a particular role in the most efficient way possible, just like the silicon retina. And this will have the beneficial side-effects of greatly improving reliability and security. Specialize, specialize!