Consider a one-dimensional random walker starting at the origin. We know how many steps the walker will take, and we also know the number of steps forwards and backwards the walker will perform. Is there a formula that can be used to quickly compute the exact probability for the walker to reach a certain maximal distance from the origin during the walk?
In other words: consider a binary string made of zeros and ones. If we know the length of the string, and the number of zeros and ones in the string, can we compute the exact probability, using a simple function (not through simulations or dynamic programming but an analytical solution), of the absolute value of the maximum running sum for the string (computed with adding 1 for 1’s and subtracting 1 for 0’s) during the traversal of the string from left to right?
Example:
For strings of length 3 we can have 000, 001, 010, 011, 100, 101, 110, and 111.
The probabilities are for 000 and 111 100% to reach level 3
For 001, 010 and 100 (or in other words when there are 2 zeros and 1 one) 33.3% to reach level 2 and 66% to reach level one. The same distribution of probabilities is true for 011,101 and 110 (or 2 ones and 1 zero).
For longer strings such probabilities can be represented in a table where the columns represent the number of ones in the string and the rows represent the maximal distance from the origin:
0 1 2 3 4
1 0 0 4 0 0
2 0 3 2 3 0
3 0 1 0 1 0
4 1 0 0 0 1

0 1 2 3 4 5
1 0 0 4 4 0 0
2 0 0 5 5 0 0
3 0 4 1 1 4 0
4 0 1 0 0 1 0
5 1 0 0 0 0 1

0 1 2 3 4 5 6
1 0 0 0 8 0 0 0
2 0 0 9 10 9 0 0
3 0 0 5 2 5 0 0
4 0 5 1 0 1 5 0
5 0 1 0 0 0 1 0
6 1 0 0 0 0 0 1

0 1 2 3 4 5 6 7
1 0 0 0 8 8 0 0 0
2 0 0 0 19 19 0 0 0
3 0 0 14 7 7 14 0 0
4 0 0 6 1 1 6 0 0
5 0 6 1 0 0 1 6 0
6 0 1 0 0 0 0 1 0
7 1 0 0 0 0 0 0 1

0 1 2 3 4 5 6 7 8
1 0 0 0 0 16 0 0 0 0
2 0 0 0 27 38 27 0 0 0
3 0 0 0 21 14 21 0 0 0
4 0 0 20 7 2 7 20 0 0
5 0 0 7 1 0 1 7 0 0
6 0 7 1 0 0 0 1 7 0
7 0 1 0 0 0 0 0 1 0
8 1 0 0 0 0 0 0 0 1

0 1 2 3 4 5 6 7 8 9
1 0 0 0 0 16 16 0 0 0 0
2 0 0 0 0 65 65 0 0 0 0
3 0 0 0 48 35 35 48 0 0 0
4 0 0 0 27 9 9 27 0 0 0
5 0 0 27 8 1 1 8 27 0 0
6 0 0 8 1 0 0 1 8 0 0
7 0 8 1 0 0 0 0 1 8 0
8 0 1 0 0 0 0 0 0 1 0
9 1 0 0 0 0 0 0 0 0 1

0 1 2 3 4 5 6 7 8 9 10
1 0 0 0 0 0 32 0 0 0 0 0
2 0 0 0 0 81 130 81 0 0 0 0
3 0 0 0 0 83 70 83 0 0 0 0
4 0 0 0 75 36 18 36 75 0 0 0
5 0 0 0 35 9 2 9 35 0 0 0
6 0 0 35 9 1 0 1 9 35 0 0
7 0 0 9 1 0 0 0 1 9 0 0
8 0 9 1 0 0 0 0 0 1 9 0
9 0 1 0 0 0 0 0 0 0 1 0
10 1 0 0 0 0 0 0 0 0 0 1

0 1 2 3 4 5 6 7 8 9 10 11
1 0 0 0 0 0 32 32 0 0 0 0 0
2 0 0 0 0 0 211 211 0 0 0 0 0
3 0 0 0 0 164 153 153 164 0 0 0 0
4 0 0 0 0 111 54 54 111 0 0 0 0
5 0 0 0 110 44 11 11 44 110 0 0 0
6 0 0 0 44 10 1 1 10 44 0 0 0
7 0 0 44 10 1 0 0 1 10 44 0 0
8 0 0 10 1 0 0 0 0 1 10 0 0
9 0 10 1 0 0 0 0 0 0 1 10 0
10 0 1 0 0 0 0 0 0 0 0 1 0
11 1 0 0 0 0 0 0 0 0 0 0 1

Hints: Catalan Triangle, “Gauss early years second paragraph http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss”:
http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss

As opposed to Chris Anderson ’s viewpoint, theory is not quite dead yet… but science is indeed changing. Technology’s new innovations have always accelerated new science, but today this is happening faster than ever. Besides the fact that computers are increasingly used to store and process scientific data, instrumentation to measure more things, more accurately, and more often are rapidly developed.

The instruments provide masses of new data where storage capacity and processing capacity are amazingly capable of keeping up with the pace of data generation. Hence, computational capacity is not appearing to be a limiting factor for progress towards new scientific discoveries. Google Research Datasets is one new example of computational resources available to accommodate Big Data. New data analyses and visualization tools are needed to handle the masses of the new multi-dimensional datasets from across disciplines, and that is where Peter Norvig and other machine learning campers appeared to have the most rational approach for an answer. Creative visualization solutions are also emerging, for example, a demo of trendalyzer , a way to visualize multi-dimensional data was given to those who went on the Googleplex tour.
Different sciences are at different stages when it comes to the imbalance between data and theory. A heated discussing chaired by KK about the future of the scientific method generated many interesting ideas: Generalizing about “too much data is destroying theory” might not be applicable in all cases. In physics, for example, theory is ahead. Most theories in physics cannot be tested experimentally. Experimental physics is going to receive a boost from the CERN collider , projected to produce petabytes of new data. In biology, on the other hand, instrumentation like fast DNA sequencing (of everything) are producing data more rapidly than we can fully computationally digest (at least right away). Hence, in biology, theory is lagging behind the data. In climate science, where theory has been traditionally ahead of the data, a shift towards the opposite is soon coming.

The tools used to analyze and visualize data across disciples seem to have commonalities. This should be exploited so efforts are not duplicated and progress can be achieved more rapidly. Peter Fox suggestion of X-informatics offers to build a community that will be developing a library of data analysis tools and resources that can be used across disciplines to narrow the divided between data and theory.
With all this optimism and excitement, and the observation that more things are discovered faster than ever before, time is of the essence. Global warming and the destruction of our own environment are of great concern. Our inability to increase the drug discovery rate and develop new and more successful treatments for diseases such as HIV and cancer (although great progress is being made) is another issue. Another concern was demonstrated through a 3D visualization of real-time Google searches around the world. It shows that Africa is left behind in this information age (the Information Divide). Cell phone technology that would deliver applications and access to everyone is a potential solution suggested by Joel Selanikio . This is in accordance with Google’s mission to provide computational resources for all.
Chris Anderson’s and Peter Norvig’s idea that theory will not be needed when everything will be measured is an interesting perspective. But in my opinion new theories will emerge when we will be able to better analyze those masses of new data. Not just through machine learning approaches or static network analyses but also by using advanced computer simulations and other tools that will track network dynamics. The theories that will emerge from looking at these massive and diverse datasets, collected from natural and man-made systems, are those complex systems design principles that emerge when you take a step back and look at common features shared among different systems. Speaking of man-made systems, hardware and robotics, as well as synthetic biology seems to be the next two frontiers. When we get to a point that we almost fully understand natural systems, we will always have to continually study the complex systems that we are creating.

I am lucky that my current boss and Ph.D. thesis mentor Ravi could not go and kindly suggested to the organizers to invite me. My first blog post was mainly a result of reflections from the first SciFoo in ’06, and just like the first time, this time around I learned a lot and had a great time.

When high-throughput methods uncovered that the protein-protein interaction network of yeast is scale-free, it was also found that highly connected proteins are negatively correlated with evolutionary rate 1. This means that the sequence of the highly connected proteins in the yeast interactions network is more fixed compared to less connected proteins. This makes sense since highly connected proteins commonly participate in many cellular functions and pathways that must be maintained, and as such, these proteins are under stricter evolutionary constrains. A similar observation was also reported in a more recent comparative genomics study that looked at changes in genes between the human genome and the genome of chimpanzee. The authors of this study identified that the genes mostly prone to evolve are less connected and most likely to be membrane proteins; two aspects of evolution at the network periphery 2. These observations suggest that central hub genes in complex intracellular biological systems are less flexible for evolutionary changes, whereas membrane proteins and less connected proteins are more flexible for acquiring mutations, which means that these genes are more likely to take on new roles. This concept of growth and flexibility at the network periphery is true for many complex systems in general. It is observed in social and economic networks where innovation and growth are more probable at the network periphery. An intuitive non-biological example of growth at the periphery can be given by looking at the rate of construction in and around major cities. Construction at suburbs can be achieved more rapidly than construction in the downtown of a major city where the streets, piping, electricity and buildings reached a level of maximal complexity and if not completely destroyed and rebuilt the streets and building are relatively fixed. Anther example is the core design of cars, planes or personal computers. These systems are evolving but their core looks and functions similarly today and many years ago. The concept of growth at the periphery means that if you look at a complex system, its overall core design is almost completely fixed and was achieved very early.
The concept of growth at the periphery seems to contradict the rich-get-richer network growth paradigm because hubs are usually found at the center of the network, and based on rich-get-richer, hubs grow fastest. Growth at the periphery might be acting as a check to ensure that the rich does not become too rich, and provides opportunities for the poor and less connected to also compete more fairly with the rich. This does not mean that rich-get-richer is not a driving force. Rich-get-richer works up to a point where the hubs can no longer grow. As complexity increases the complex system’s internals become more fixed and more rigid to changes and growth. As the system continues to evolve, it is mostly changing functions and mechanisms related to interactions with the environment, cosmetic changes and not core basic functions. Changing the internal structure is often too costly, risky, or impossible.
Growth at the periphery is linked to the concept of life-at-the-edge-of-chaos. Stuart Kauffman Boolean dynamic simulations in the early 70s’ helped him realize that life has to exist at a fine toned zone between complete freeze and chaos. Simulations of Boolean networks and cellular automata showed that interesting patterns, similar patterns observed in natural life, can be reproduced by simple dynamical models when the dynamics are set to be just right at that edge between order and chaos. Here, with growth at the periphery, the edge-of-chaos is at a circular periphery of the complex system’s network giving the edge-of-chaos concept a circular geometry.

In the movie antitrust, spies from a giant software firm placed hidden cameras behind the shoulders of open-source developers to steal their source-code. Company representatives then murdered those programmers to give their company a competitive edge. This is an extreme example of drafting; one of those complex systems’ design principles. The word draft has 38 definitions on dictionary.com , but drafting here is used in the context of slipstreaming, definition #34 that states: “…ride close behind another car so as to benefit from the reduction in air pressure created behind the car ahead.” This concept is not only true for car racing or cycling, in a more abstract sense it is one of those design principles driving commonly observed behaviors in complex systems. Drafting is related to innovation, rich-get-richer, duplication-divergence, energy efficiency, and reproduction. Drafting, duplication-divergence, and rich-get-richer are directly tied to some measure of success. There is no point in duplicating a product, an agent, or a process if they are not successful. There is no point to draft behind a loser. When a product, a process or an agent in the complex environment is gaining some competitive edge, or when an agent’s fitness is increasing, which can be measured by overall proliferation or popularity, those who have the resources, the big hubs, the listeners-to-innovations, would soon create a replica and swallow the smaller innovative fish. This is different from other rich-get-richer dynamics such as followers lining-up behind a new leader trying to mimic its behavior in hopes to become a leader themselves one day, as well as be protected from potential isolation and elimination. Just like the giant software firm in antitrust, the powerful listeners “sit” right on the shoulders of the innovators to “collect” the fruits of new ideas. The web, open-source software, and the open-source movement in general, including this web-site, are changing and challenging antitrust and drafting dynamics in both social and economic networks. With few exceptions, “giving-credit-where-credit-is-due” social justice has been exercised poorly throughout human history and this may be changing. What role “drafting” will play in the future on the web? Will “drafting” increase or decrease due to the open-source movement? In economic and technological networks “drafting” is easier and faster to achieve and to be detected. In comparison, “drafting” in biological systems overlaps with the concepts of duplication-divergence, rich-get-richer, reproduction, and energy utilization making it a bit harder to define.

Big Data offers an opportunity to observe that complex systems, natural or man-made, share many universal design principles. The cell, multi-cellular organisms, ecosystems, economic systems, societies, intricate engineered systems, and the web are all evolving complex systems existing in complex environments, and all sharing common design. Complexity theory researchers often focus on studying only one design principle, mostly applied to only one real-world complex system: ironically, still reductionism. It may be insightful to look at how design principles of complex systems are related. To develop intuition about this idea, observed principles can be assembled and organized, and then the relationships between them identified. An initial list of design principles observed in complex systems with some hinted relationships follows:

Survival of the fittest is a central design principle shaping complex systems. This concept is an outcome of competition. Competition is often not fair. Rich-get-richer is a growth process where the rich, the one having many relationships, central, essential, and fit, grows faster than the poor, lonely, unfit, weak, and less-connected. Growth in complex systems is often achieved by duplication-divergence. This known biological design principle is also common in economics or on the web. For example, car models, web-based companies, and software products grow through duplication-divergence, and where innovations play also an important role. Although growth is rapid for the rich and central, rapid growth at the periphery is also observed. Another essential concept is information transfer. Information is constantly flowing, commonly compressed, decoded, and translated. Information is commonly intercepted by sensory units. Sensors transmit the state of the environment into internal processing centers. In those centers information is processed intelligently by classifiers that learn and use memory to determine an appropriate response. Often this response is simply turning on or off a switch. Sensors often implement filters and amplifiers to convert noisy information to valuable data. To compute the right response, processing centers use learning, memory, and adaptation. The ability to adapt to new environments is critical for survival of agents in complex-systems. Robustness to fluctuations and changes in the environment is required for fitness and survival. When learning is successful, responses are often automated. Automation is also used for efficient production. Efficient and sophisticated mechanisms are in place to manufacture many exact replicas. This allows the cycle of birth-life-death to continue. The birth-life-death concept is related to the observation that agents are dynamically replaced by new agents while global patterns remain. For example, proteins in a cell, water molecules in a river, cars on a highway, blood cells traveling through blood vessels, or people commuting back and forth to work into a big city. In some cases these agents circulate while in others completely replaced. Agents often travel on elaborate and efficient transportation systems that permit the transfer of resources to remote locations quickly. To move around, locomotion is required. Locomotion is the ability to travel. Economic systems rely on planes, ships and trucks to transfer goods and workers. Botanic plants lack the ability to move. This handicap is compensated with an amazing ability to use solar energy, to extract nutrients from the ground, and to reproduce effectively.

Noticeable barriers are present to protect agents from other agents and the outside. These containers or modules hide internals from exposed externals. The externals have an interface, often able to communicate with the environment and listen to it, as well as communicating with other agents, using standard protocols. The plug-and-play design principle allows reusability and generality which permits agents to work together to form higher-order agents. This design principle is related to the ability of many agents to switch between individual behaviors to pack mentality. When in a pack, agents form distinct geometrical patterns such as trees, waves, and other symmetrical shapes. Polymorphic agents in a pack behave randomly in parallel but often display amazing synchronicity. Synchronicity can be achieved through governments but often not requires them. Randomness and noise is required for emergent behavior, where it is used to overcome local minima. Randomness and noise result in a constant search for equilibrium, but the system never settles at equilibrium forever. Phase transitions happen in short time periods where a system, being in a rather stable state, go through one small change that induces many changes, turning the system into chaos or into another new stable state. Searching for a lower energy state is a design principle directly related to efficiency and energy utilization; where many processes in complex systems use energy, and where agents compete for energy resources. This goes back to the first concept of competition and survival of the fittest. Feedback loops are important dynamical structures that set the creation of complex systems in motion. The primordial metabolic soup was suggested to be made of simple enzymes forming competing feedback loops. Competition is taking action in markets, where trade is used so all are winners. Market fluctuations often display pendulum behavior shifting from side to side. Successful trade can be achieved when there is diversity and heterogeneity, and where the winners are often the innovators, or the drafters, the best listeners to innovation. Trade also results in cooperation. Cooperation can develop to symbiosis: the codependency of two separate agents on the coexistence of each other. Unidirectional symbiosis is the emergence of parasites. These agents use the success of their hosts for their own survival. Successful agents must learn then how to self-repair and fight parasites, while engaging in a game of creative evasion strategies with parasites, because parasites fight back.

As we are able to create complete independent autonomous agents, living in new self-sustaining environments, where biological systems can be more engineered, and engineered systems are becoming more complex. By understanding how design principles of complex systems are related, we should be able to better understand, control and manipulate complex systems.