Swarm Intelligence and Collective Memory

(Firstly, I have nothing new to say about the attacks than you have probably been reading around, except that the Spirit of Mumbai is not about how indifferent we are to the attacks; rather it is about how sensitive and responsive we are to the need of the hour)

Swarm Intelligence, Collective Memory. These two ideas.. what do we know about them? I keep hearing about Swarm Intelligence quite often in the context of robotics, mostly; and quite frankly I was unaware of the idea of Collective Memory until I saw the movie Waking Life.

Yet the two have a very natural and deep connection that escaped me for a long time, until only very recently, when my research problem came down to actually closely observing the behaviour of a swarm of “pseudo-intelligent” beings.

Pseudo-intelligent, because they are only as good as the C code that Created them. (And just like the ‘real’ God, the same code will also have to responsibly Destroy them; thanks to the no garbage collection in C :D) But other than that, these particles actually have pseudo-life; for as long as they live, they literally swim around in space, talk to each other, and split into more of themselves; looking all the time for something.. let’s call it cake, for the sake of discussion.

But the most important aspect of this method (called Particle Swarm Optimization(PSO)) is this: the advantages of PSO start appearing only when the swarm collaborates, exchanges data and puts together all information it has about the cake. In other words, the particles must start making use of their Collective Memory; as opposed to their individual, independent thought.

So, in short: A swarm’s intelligence is as good as their collective memory

Simple as these ideas are, we see manifestations of them all around us. For example, the concepts of Cloud Computing and Grid Computing seem to be conflicting, since one of them is about centralization and the other is about decentralization. Yet, they not only co-exist, but also influence each other’s prowess. (Because after all, the ‘cloud’ IS a grid). Another related and interesting concept is the emerging digital currency called Bitcoin, which is backed not by any Bank or Government, but actually by the network of the many Bitcoin users collectively performing many computations even as you read this.

To conclude, I suspect that the thing I said about swarm intelligence being as good as collective memory, is as relevant to us as those software particles; but not all of us consciously act accordingly.

14 responses to “Swarm Intelligence and Collective Memory

  1. When you talk about a swarm sharing information in the sense of a collective memory, it is like a central log of whatever the swarm system knows all together, right? The next thing is how the swarm system as a whole can make use of this collective memory to expand their intelligence, in the sense of “understanding” all this data and acting on it. Because this *is a swarm*, you should’ve defined properties only at the individual level and interactions between these particles. In this context, can you please elaborate on “A swarm’s intelligence is as good as their collective memory” from an informational theoretic perspective? And from a “understanding this data” perspective, if that makes any sense to you?

    • So, the swarm is actually ‘swimming’ through the domain of a function that is to be optimized.. hence from an informational theoretic perspective, each particle has information about the value of the function at the particle’s current and previous locations. Now as such, if absolutely nothing were known about the function, then one simply cannot predict where the optimum point would be, based on this. But given ‘nice’ things like continuity, and bounds on the max gradient or second-order speeds/profile of the function, then the collective information that the swarm has about the function’s values at various points, should be enough for the swarm as a whole, to guess in which region the function is likely to be smallest in value, and then the whole swarm can start exploring that area/direction.. and so on iteratively. So although programmatically the swarm’s “intelligence” does not change over time, what does change is how much the swarm can achieve from whatever it knows. And this “how much it can achieve” is governed by “how much it collaborates”.. there are inherent trade-offs in collaborating too much, BTW.. and this in itself becomes an interesting optimization problem!
      PS I hope this is what you wanted

      • I’m not sure what I wanted, but this is *something*! Each particle has local information, or rather local memory (in your illustration, a length of two), about the value of this objective function at the points on the domain it has moved through. Now each particle moves to a new location according to some interaction and update rules (say IU), I suppose. IU might implicitly contain these “nice” characteristics of the objective function I guess.

        “the collective information that the swarm has about the function’s values at various points, should be enough for the swarm as a whole, to guess in which region the function is likely to be smallest in value, and then the whole swarm can start exploring that area/direction.. and so on iteratively. ”

        In the context of these lines, the collective information is not a global thing that every particle can access, right? For each particle, one could have a spatial neighborhood (over the swarm space) with which there’s active IU. Is that the way it works? All this shifts results in an oriented and emergent shift in the densities of the swarm particles towards specific points in the domain – max/min points as determined by the specific IU one uses.

        Talking of tradeoffs in the extent of collaboration, is it related to suboptimal convergence to local extreme rather than global, or rather a smaller basin of attraction for the global extrema in a systems context?

        • Yes, IU contains the niceness.
          So as such in the specific case of my problem, the collective information has in fact been made globally available to all particles; since in a RAM card, there is no question of ‘distance between particles’ being a constraint 🙂

          And yes, the trade-off is related to suboptimal convergence as well as taking too many iterations/computations to locate the optimum point. And regarding the smaller basin of attraction, the given niceness about the function helps one to predetermine how densely the swarm should occupy the domain, so that the basin is not missed.

          • Really interesting. It would be interesting to study the effect of localizing information on the convergent characteristics of the swarm system. Intuitively, we can expect better exploration of the domain when information is locally shared (but the sharing happens everywhere, of course). Nice stuff! Food for thought.

            • Ah.. you mean if the information is centrally decentralized! (Or better still, globally localized!) Food for thought, indeed.
              In particular, an immediate advantage of this would be that each particle needs to process information of only a small subset of the whole swarm; and this directly reduces the computations by a decent margin

              • The globally localized approach has been explored a lot I believe. Yes, there’s an obvious computational gain but this might happen at the expense of slower convergence. Increased robustness to stochastic variations (and possibly robustness in other contexts as well) seem to be an advantage of making IU globally local. I’m extending ideas I’ve read from elsewhere, especially complex networks, and adding a layer of an {objective function and convergence} to talk about our (or rather, your) problem.

  2. Maybe you will enjoy one of my papers on the field: Vitorino Ramos et al. Social Cognitive Maps, Swarm Collective Perception and Distributed Search on Dynamic Landscapes, submitted to A. Porto, A. Pazos, W. Buno (Eds.), Advancing Artificial Intelligence through Biological Process Applications, IDEA Group Inc., 2007. LINK: http://www.chemoton.org/ref63.html Kind regards, Vitorino

    • Your paper is indeed a very interesting read! I’d be very curious to try and see if some of the ideas described there can be applied to the problem I am working on. Thanks.

  3. Pingback: Swarm Intelligence: Is the Group Really Smarter? « People-triggers

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s