Clairvoyance

Clairvoyance

On the morning of August 10, 2045, Devesh logged into his computer, hoping that the computation he had set up a few hours ago had finished. He was proud of this new computer of his. True it wasn’t sleek or elegant, and it was several gigs short of the best computing machines of the day, but it allowed him to do what his decade old desktop never could. He had downloaded now the latest version of Ekon, one of the best general pattern analysis software that one could get over the internet. He wasn’t very familiar with programming, even though his college degree had had ample amounts of it, and he was bad at math, but he knew enough to be able to use it reasonably well. Besides, in his journalist profession he didn’t really need accurate numbers; that was only for the experts.

He only used his computer to get a rough idea of where the world was headed. Today, for instance, he had decided to see if the government’s proposal to construct an Industrial Zone in the Giri tribal region made any economic sense. In the good old days, he would have had to phone up some economics professor in the university and try to prod out of him any quotes, or ideas, or remarks that he could expand into a story. But the world was different now. The subjective opinion of a half a century old professor who did not read the news was far less valuable than computer programs that could analyze the trends of GDP, GNP, indeed, income levels of every village and block in the country and tell you what you wanted to know. What’s more, you did not even have to call anyone up and fix an appointment.

Well, he thought to himself, as he sipped his coffee, you probably still needed an expert to make sense of all the numbers his computer was spewing. You had to know what to look for. But he could look at the graph and see that investments would grow, and the economy growth rate would be given a small boost. Perhaps the government was right. Plus, the Giri region was one of the most backward in the country. It would probably help the region’s economy quite a bit.

There was the odd lobby that was afraid, or angry (he never could decide which) that the people in that area would have to be relocated. But he didn’t have to be an expert to see from the numbers that the inhabitants were impossibly backward anyway. A move to the city should do them some good. Maybe he should add that as an addendum. Adequate compensation should also be paid to the families.. or something more flowery, perhaps.

***
On August 10, 2045, in the village of Nallur, the sun beat down with venomous force on the red soil from a cloudless sky. The gathering at the chaupal  stood silent, and despair hung heavy in the air. A man stood in the centre of the gathering, holding in his hands a small phone; he had just finished a survey of the farmlands in the village.

“It is just as was predicted last year”, he said again. “The forecast for the next couple of years seems very bleak too and there is unlikely to be much rain this year either.”

He paused. He took pity on the villagers. For many of them this was the only source of livelihood they were capable of; they knew nothing else. But every year that the rainfall was scant, the chances that these villages would come out of famine just kept getting weaker and weaker. He had seen the trends at the block headquarters, they were unmistakeable. He didn’t have to be an expert to see that farming in this area was not tenable any more.

“As I said, all of you have to understand that you have no future in this land. This place is not going to get better. The government is planning to construct an industrial center here; in return for your land they will not only give you money but also jobs in the factories that will come up. It is a much better option.”

A few voices of protest went up, but were quickly subdued. He sighed. It was hard for them to part with their land, but there was no alternative. This was inevitable. The crowd hung its heads in silence, while overhead the afternoon sun went on relentlessly about its day.

***
By the end of that day, both these stories were merely data on the internet. Information about the agricultural yield in the Giri region percolated up through the block, district and state networks and was finally stored in a huge database at the center. Devesh’s story too went through his editor’s table and onto the web page of the Times Globe in a couple of hours.

Beneath the central government’s residence in the capital, a huge stack of servers whirred incessantly, as they looked at these, and all other possible sources of data, sifting through the massive flow of information and trying to predict trends and optimal choices. Devesh’s story was quickly analyzed for its opinion on the Giri Industrial Zone; it was considered as mildly positive and stacked away. So were hundreds of other blog posts, news articles, videos. The agricultural yield from the region was also taken into consideration, as an impartial algorithm noted that the yield was low, as predicted, and that the per capita income of that region continued its downward trend. The feasibility of the Giri Industrial Zone was considered again; it was still deemed feasible. The estimates of profit were slightly adjusted. A report full of trends and numbers was prepared automatically.

The next morning, analysts at the Ministry would log into their computers and see the reports. They would understand that the Giri Industrial Zone was a go. The Minister would be advised appropriately, he would sanction the project, and the wheels would be set in motion. New analysis would be done the next night, depending on what the government wanted studied. It would probably not be anything to do with the Giri region though; already by the night of August 10, the future of Giri was sealed.

***
“I really appreciate you bringing your daughter here, Mrs. Sharma”, said Mrs. Khan. “Iqbal doesn’t seem to play with any one else.”

The lines of worry were very prominent on Mrs Khan’s face.
“What did the doctor say?”          
“They don’t know what it is”, she said, throwing her hands up in exasperation. “They say that he  might be autistic, but it is not that he has trouble communicating or anything. He is perfectly fine, he talks to me about what happened in school, what the teacher said and all. But he doesn’t understand basic math. Every time I try to explain to him addition or subtraction, or ask him to recite the multiplication tables, he just goes blank. I don’t know what to think.”


“Some children have problems paying attention”, Mrs. Sharma ventured.
“But that is not the case either”, said Mrs. Khan. “He does pay attention other times, you know. Like when he is drawing. You should look at him when he is drawing. He won’t hear you no matter how loudly you call his name.”
“He does draw very well..”
“Yes, he does. He has a funny way of drawing too. I told that to the psychologist, and he asked Iqbal to draw something in front of him. The psychologist was just perplexed. He would have said that he hadn’t seen anything like it had he not been so intent on appeasing me.”
“Does he still do those...strange things?” Mrs Sharma asked carefully. Mrs. Khan nodded.
“It isn‘t frequent. The other day we were waiting for a bus, and we were standing beneath a gulmohur tree that was in bloom. And Iqbal just looked up at it once and then shook the tree so that a flower fell almost precisely in his hand. I mean, it wasn’t as if it fell accidentally. He had his hand outstretched, he knew where it would fall. You should have seen him.” She shook her head slowly, “When he does things like that I just freak out..”
“There, Mrs Khan, I am sure Iqbal is a perfectly normal boy.”
Mrs Khan sobbed. Mrs Sharma held her hand and looked at the boy, who was sitting on the ground with Jyotsna. He was drawing something, and Jyotsna was looking at him drawing, even as she kept talking incessantly. She couldn’t understand what either saw in the other, but for Mrs Khan’s sake she was glad that Iqbal had found a friend.

In spite of herself she looked at Iqbal drawing. She had to admit it was funny, the way he moved his hands all around the paper, hardly completing one stroke before moving to the next. It was quite unlike what she usually saw children do. There were no triangular mountains or blob shaped trees, or rectangular houses. Iqbal’s drawing seemed to have an incredible amount of detail. Even though he spent little effort in drawing the mountains or trees, they seemed incredibly realistic. Mrs Sharma found herself wondering what it was about the drawing that made it so beautiful, and what it was about Iqbal that made him so different.

***

“I don’t understand how you do it”, she said, folding her arms across her chest. They were sitting beneath a tree beside a stream. The sun was warm and strong, but it was cool and pleasant in the shade.
“Do what?” asked Iqbal, throwing a pebble into the stream. It bounced off the water several times before sinking into the water.
“What you just did.”
“You mean bounce pebbles off the water? You can do it if you try.”
“Not that, I mean that”, she said, pointing to a pebble resting precariously on a smooth rock in the middle of the stream. “It bounced off three other rocks before landing there. You put it there didn’t you?”
Iqbal shrugged and smiled. “You wanted me to.”
Jyotsna ignored his answer. “Of course you could have calculated it all. It is all science”, she said knowingly. They were fifteen years old now, and only yesterday they had started tenth grade, getting introduced on their first day to Newton’s laws of motion. “You could have figured out all: the force and acceleration and velocity and everything.”
“Well I did not”, said Iqbal haughtily. “None of that makes sense anyway.”
“No you did. There is no other way that pebble could have landed there. You just don’t want me to know.”
Iqbal laughed. “Really? And why is that?”
“I don’t know”, she replied, “all I am saying is, that you are not a psychic or something. There is some scientific thing behind what you just did, and I am going to find out what it is.”
“Sure thing, and let me know when you do. Meanwhile, care for a pebble?” he said, handing her a pebble. She took a pebble and threw it absently at the stream. It bounced off the water a couple of times and then collided Iqbal’s pebble resting on the smooth rock. Both the pebbles fell into the water with a plonk.

Jyotsna and Iqbal looked at each other and laughed.

***
He stood in front of the sarpanch as the latter looked at him despairingly.

“It is not going to work, Sir”, said the sarpanch. “You are not the first person who has tried. The weather just keeps getting hotter and drier every year. You talk of the water that flows down the hill but it is so little that it is of no use to us. We also have these rain water harvesting systems that the government installed several years ago. They collect water, but never enough. The wells are as dry as before.

“And they have analyzed the trends perfectly sir. The situation is not going to improve. It is only going to get drier. Many of us have already made plans to go to the city in any case, even if the government doesn’t build the Industrial Zone they keep talking about. It is of no use.”

He sat down beside the sarpanch and tried to convince him. “It is not too much I ask of you, Sir. I can pay you money, in return I ask only that you allow me to do some work on the hill sides of the village. Think of it as one man’s whim. You will soon see what I meant to do.”

“But what can you do when this is our fate?”

This was one of the elders of the village. He sighed. He looked around at the faces of all the villagers around him. They were grim, despairing. Around the village the red soil was baked, hot and dry. The sky was cloudless as always. There wasn’t the slightest bit of moisture in the air.

How could he make them believe in the future? How could he stop them from giving up all hope? Perhaps it was already too late. Perhaps his efforts were futile already. Still, he had to try.

The next week he managed to recruit some of the village youth to work with him. To them this was just a job. Better than working on barren land, they told him, shrugging. He sighed, but he had no other choice.

They dug up trenches around the hill. He told them where to dig, how deep to dig. They planted saplings beside the trenches. The villagers asked him how the saplings would grow with no rain, but he didn’t answer. He knew the rains would come just in time. He had tried to talk an NGO a few hundred miles away, who had done considerable watershed development work, into doing this. They would have had the right kind of expertise for this job. But when he had suggested the idea they were nonchalant, even dismissive. They had simulated it on a computer and decided it was not worth the effort. They had given up too.

Iqbal didn’t know how to make them realize that it wasn’t about the water. It had never been about the water. It was about hope.

He sighed and began digging another trench.

***

The CEO of United Technologies walked into the board meeting feeling thoroughly uneasy. He rehearsed what he was going to say in his head, but it still felt marvelously incompetent. The thing was, he did feel incompetent.

After they had won the bid for the Giri Industrial Zone project, they had to raise capital by issuing shares. But the whole idea had gone bust. For the past month or so the stock market had consistently gone down. The funny thing was that no one seemed to know why.

“How are we performing on the stock market now?” he was asked as soon as he stepped into the room.
“Still going down. We are down to less than ten bucks a share.”
“Do we have any idea what is happening?”
“No, not yet. We are still trying to figure out.”
“I thought it was easy to figure out what is wrong. What do the regulators have to say?”
“They don’t know either. It is strange. This crash seems to have slipped past all the safety nets. No one seems to have predicted it.”
The people in the room stared back blankly at him. “I don’t understand you, Frank. You are talking as if this is the 2000s. This is 2045. You always know what is happening to the stock market.”
The CEO ran a hand through his hair. “I am not sure how to explain it. It is true that there is a huge data center in the capital that continuously analyzes all the transactions, but word from there is that they still haven’t nailed down what caused the crash.” He paused. “There is a small probability that there might have been malicious intent behind this.”
“What do you mean?”
“Word is that there were a few transactions that were suspect. Only a few transactions, which were ignored since they were so few in number and didn’t involve big amounts. But somehow they caused a cascade of events that crashed the stock market.”
There was silence in the room. Someone else spoke. “How does this impact us? Do we have enough capital for the Industrial Zone project?”
“Sadly, no. We don’t have money enough for that. That project would have to be put on hold.” He coughed. “We might have trouble paying our older loans too.”

There, that was it. Bomb dropped.
The room was silent enough for him to hear his own breath. He could almost feel the blade of an axe on his neck.

***

Jyotsna’s eyes shone as she talked. Iqbal wasn’t really getting what she said, but he enjoyed looking into her eyes when she was in one of these moods.

She paused and smiled. “Are you listening to me at all?” Iqbal laughed. “Not really. I don’t really get all this math.” Jyotsna sighed. “It has been, what, nineteen years since I first met you and I still haven’t been able to teach you math”, she teased.
Iqbal shrugged and smiled. “I don’t know why you tried.”
“But seriously, listen”, said Jyotsna, “this is about you too.”
“About me?”
“Yes, I know how you can do those things.”
“Which things?”
Jyotsna threw her hands in the air. “as if you don’t know. Your apparent clairvoyance, that’s what. The things you seem to predict so easily.”
“So you managed to get your computer to out-predict me?”
Jyotsna laughed. “No. I just took a bunch of different situations and simulated them in excruciating detail.”
She paused. “And?” asked Iqbal.
“It seems”, she said, thinking of how she could explain it to him. “It seems the common thread in all that you manage to understand so easily is the presence of chaos.”
Iqbal smiled. “Yes, I am enlightened.”
“Okay, okay I am explaining. In very simple terms,” she said, “when a bunch of things interact together in a non-linear fashion, they end up becoming apparently random. If one is trying to predict trends in such a system, small errors get compounded again and again till they become huge, so looking at it mathematically just makes it hopeless. The kind of errors that many of these statistical tools are meant to tolerate just get compounded, till any sort of prediction becomes totally useless.
“But for some of these cases, there is an underlying structure to the randomness. A very different structure, that is hard to detect, unless you stand back  a little and look at the big picture. Like water molecules coming together in a cloud, or in a snowflake. There is a lot of complexity in how they come together exactly that makes it really hard to simulate, but the broader structure is clear as day.”
Iqbal nodded. “And where do I come in?”
“You”, she said, thoughtfully, “seem to somehow see this structure. You seem to be able to make some sort of prediction even when the best computer is bogged down by complexity.”
Iqbal smiled. “I thought you said I wasn’t clairvoyant. That seems all magic-y to me.”
Jyotsna shook her head. “It is not magic really. We have not gone very far in this area yet, but it is mathematically well defined. Also these structures pervade our life sufficiently for me to believe that all of us might also have your ability to some extent. I mean, we have a huge repertoire of intuitions that don’t really follow conscious reasoning. For all we know this might just be one of the many intuitions we have developed for surviving in this world. We just don’t know about it yet. Maybe you are a special case who has way more intuition than any one of us.”
Iqbal nodded thoughtfully.
“So”, he started, “the tools you build, do they also try to predict these....structures?”
“Not really. As I said they are way too hard to model right now. We just ignore the big picture and treat it as random.”
“Hmm,” said Iqbal thoughtfully.

***

The story of Nallur spread rapidly among the surrounding villages of the block.

When the people saw lush green crops covering what was once thought to be barren and sterile land, they could not help but be spellbound. They were told the miracle stories. How the rain was still scanty, but had somehow been enough. How the wells had started to fill. There was only just enough water to grow one round of crops, but that one round of crops was enough to set the ball rolling. The sarpanch, excited about the crops had told the surveyor when he had come to survey the agricultural yield. The surveyor, amazed, had noted that the trenches on the hillside, and wrote a report praising the watershed management that had gone a long way in turning the fortune of the village. He had spread the word to other villages. The news had spread like wildfire in the entire region.

When experts from the NGO, and later from the district administration, came to look at the watershed in Nallur, they were amazed. It was a miracle. All their calculations had shown that there was way too little water and the watershed development program would not work. They decided the year was a fluke. The rains had been good this year, the next year would not be so lucky.

But to the villagers this hardly mattered. They saw it as their victory against fate. They had been told that the famine would never end, and here they had almost a full crop. There was still a  chance, they thought. There was hope.
When the neighbouring villages saw the change, they built watershed programs of their own. The experts shook their heads, but the youths of the village went ahead and planted saplings and bushes along the hill slopes, built trenches to stop water runoff, and went through the whole program of watershed development that Iqbal had put Nallur through a year ago.

The next summer came, and then another. The agricultural yield out of the Giri region quadrupled. By now the shock waves reached the state government headquarters. The chief minister wanted to know what had caused the miracle. The entire region was studied again. The answer, they found later, was breathtakingly simple.

The change that was too small for one village was huge when it was done by all the villages put together. Now the entire Giri region was one big watershed; the saplings planted by each village on the hillsides meant a massive rapid afforestation drive. The western slopes of the Giri mountains went from being barren rock and stone to row upon row of lush vegetation and tall trees bearing fruit. In a few years the geography of the whole region was firmly on the path of change.

The chief minister was amazed that the predictions of all the analysts he used to consult had changed this rapidly in the past four years. He thought of it as a miracle.

The villagers, however, thought otherwise.

Many analysts saw the difference, and understood what was coming.

***

Jyotsna threw the paper down on the table and paced to and fro.
“They just don’t get it. They refuse to admit it. Just refuse.”
Iqbal smiled. “Don’t work yourself up. They will get it. They will listen to you.”
Jyotsna paused. “You are planning something, aren’t you?”
Iqbal laughed. “You can predict me very well, it seems.”
“Come on, out with it. What are you planning?”
“Well, it is rather a big plan.”
“I am listening.”
“It involves playing a little on the stock market. And visiting a village or two.”
Jyotsna looked at him questioningly.
Iqbal began to explain.

***

[
Aug 10, 2049

The events that have unfolded over the past few years in Giri are, at first sight, merely an evidence of bad and hasty planning on the one hand, and the remarkable virtues of watershed development on the other. Neither of these is new. The government has made many mistakes in the past, and the benefits of watershed development form a standard component of any high school geography textbook.

Yet, if you look closely, the Giri issue is a chain of several interesting events. Let us first recap the facts. In 2045, Giri was a barren, backward land with an economic output of practically zero. The government made the decision of building a major Industrial Centre there instead, which as I said at that time, was a prudent idea. United Technologies bid for and got the project.

Then the first of two oddities happened. The stock market crashed in 2045, for unexplained reasons. The standard story now is that a few transactions caused a whole cascade of panic selling, causing a delibitating crash that left many companies reeling for the next couple of years, most notably United Technologies. However, the point to note is that in spite of several million dollars invested by the _ regulator in keeping the stock market under check, this crash managed to escape all the safety checks. In other words, none of the high-tech analysis tools used by the financial market had any idea what was happening.

The second, lesser known but parallel, oddity happened, when several experts agreed that Nallur’s watershed program wasn’t enough to avoid the famine, but the combined efforts of the entire region unexpectedly succeeded, with results for all to see. And again, and equally strangely, a slew of well tested analysis failed to predict Giri’s surprising turnaround.

As we all know, the upkeep of these two events was that Giri suddenly became relatively prime agricultural land, and the villagers refused to give up their land. And United Technologies could never recover from its losses to go on with the project, with the result that the Giri Industrial Project was scrapped.

The question, however, is this: why were our initial expectations so wrong for both these events?
In the past half a century we have come to rely extensively on statistical tools to analyze data. With computational power becoming cheaper by the day, it has become  almost trivial to look at recent trends and make forecasts about the future. Yet, clearly, these methods that we put so much trust in, are not as infallible as we assumed. Indeed, in a paper published in 2045, the noted statistician Jyotsna Sharma, then a graduate student, suggested that state of the art pattern analysis techniques could go completely wrong in certain situations. To quote:

..While techniques that have been developed so far in the literature do extremely well for predicting isolated, independent events, all else kept equal, it has to be bourne in mind that once deployed in the real world many of the underlying assumptions are no longer true. ... In the real world, everything affects everything else, and there might be umpteen sources of information that have not been modeled at all, and that cause errors that are systematic, deterministic and extremely significant.

Even more seriously it is no longer true that the algorithm can choose to make decisions independently of the outside world, because decisions will be taken on the basis of these algorithms, and the decisions will subsequently affect future inputs. Indeed, when such an analysis tool is part of such complex dynamical systems as are in the real world, it is unclear how they will respond, and how valuable the forecasts will be ... What this means, in effect is that while all of these techniques perform perfectly well in a laboratory setting, they can be prone to huge glaring errors in real life...Such errors might be rare, but they do mean that no analysis can be considered as the gospel.[emphasis added]

What Dr. Jyotsna seems to suggest here is that statistical analysis, in the way we use it, might end up ignoring factors that were crucially important; like the first few transactions in the stock market crash of 2045, or, in Giri’s watershed program, the fact that all villages were executing the plan simultaneously. In both these cases, as Dr. Jyotsna suggests, our forecasts would be systematically and significantly wrong. Talk of foreshadowing.

But Dr. Jyotsna also makes another point: that when we base our decisions on such analysis in the real life, and when these same decisions form the basis of future analysis, we become a part of a feedback loop. For the Giri farmers, this was the cycle of despair and famine that they were trapped in. The forecasts that Giri would not rise out of famine had become, for them, a self fulfilling prophecy.

Giri’s turnaround required the farmers to stop believing in trends and numbers and take their future in their own hands. I am not a spiritual guru, but there is probably a lesson in here. Perhaps it is just as well that we cannot forecast the future exactly. Perhaps we should not strive to. For what is the point of knowing a future that you cannot change?
-Devesh
]

Jyotsna sipped her cup of coffee as she finished reading the article. She was mildly amused. This wasn’t the sort of publicity she had intended. Well, she did not mind.

Her cell phone rang. “Is this Dr. Jyotsna Sharma? I am speaking from....”
She smiled. It had begun.