April 22, 2014

2014 : An Agenda for India

The month of May will ring out the old government and ring in the new. Irrespective of which party wins and who gets the right to address the nation from the ramparts of the Red Fort in August, what we need is a vision, a roadmap, a set tasks and milestones that will help chart the way to peace and prosperity of this nation. This post will define the contours of such an agenda by identifying three major areas and will then propose solutions based on available and cost-effective technology. There are many areas where India lags behind the rest of the world, for example gay rights, environmental hazards and so on, but here we will focus on some three key areas that, if addressed adequately, can lead to automatic improvement in all other areas.

Seal from Harappa 

First is education -- a nation consisting of people who can read, write and are aware of their surroundings can solve many of its problems on its own. Which is why our political classes, despite the much hyped RTE, has kept our children ignorant. Second is corruption -- because each and every step that we take gets bogged down by the rampant theft of money. Finally we need speed in our governance. Government processes must be made to move faster so that actions can move from contemplation to completion. Education, Corruption and Speed - that should be the new mantra.

Education :


The highest priority is primary education. There are 680 districts in India and we must set up 3 new schools in every district within the next 5 years. 2000 schools at Rs 25 crores each (estimated cost, actual will be less) means Rs 50,000 crores or Rs 10,000 crores per year. This is not a big sum of money -- the CWG cost us US$ 4.6 billion = US$ 4600 crores = Rs 2,76,000 or 5 times as much  for just a 12 day 'festival' -- but even this can be raised very easily through tax free bonds. However we need an ace engineer like "Metro Man" E Sreedharan to build an organisation ( for example, see  my post Shikshajaal)  to ensure that the project delivers 2000 schools in 5 years without delay and cost overruns. As a collateral, this will generate employment for teachers.

College education need to be freed from the tender mercies of pointless bureaucracies and corrupt license-sellers like the AICTE and UGC, who, in collusion with crony capitalists, have ensured that private entrepreneurs cannot deliver quality educational services to the population. Instead we must model the education sector on the corporate sector : strip the the AICTE / UGC of its role of accrediting colleges and convert them into the equivalent of a Registrar of Companies that enforce regular audit and mandatory disclosure. Based on this information, let the market, that is students and their parents, decide which college delivers value for money and they will flock there as in the case of the stock market or any service like banking.

Finally, existing government colleges can be linked in with digital technology and be converted into platforms for online education and distance learning. This will lead to better usage of scarce resources, like good teachers, and carry higher education to the distant corners of the country where it may be difficult to attract good teachers.

Corruption


Corruption is endemic to the government if not a pandemic across the nation itself. Years of cussed cynicism has made us, as a nation, willing participants in or helplessly indifferent to rampant theft of public money, goods and services. We have the Police, the CBI, the CAG, the Vigilance Commission and what not and yet each of these agencies are staffed by people like us and hence there is no guarantee of honesty. The medicine itself is fake and poisonous.  The Lokpal ( or JokePal ) will be no different. Instead, what we need is an utter, torrid, brutal and automatic transparency that is built into each and every financial transaction that involves government money.

First why wait for an RTI process. Why is it that the audited accounts of each and every organisation that deals with the disbursement of public money is not mandatorily placed in the public domain ( in XBRL format, on the internet ) so that anyone, including automated BOTS, can scan through the mountain of data and discover questionable deals. Taking this one step further, we can even bypass the formal audit part and mandate that any transaction that involves the transfer of money between public and private entities must ( after anonymising for privacy ) be made public on the web. As explained in an earlier post, such a task is far simpler that either the ill-fated UIDAI project or the reasonably successful Golden Quadrilateral project and can be executed with the minimum of fuss and effort. Using "big data" and other data analytics techniques, this huge mass of data can be analysed to show how every rupee of public money is moving through the system and then anyone -- not just the CBI, CAG, Vigilance -- can keep track of the money and detect and prevent theft and leakage.

Speed 


Our government agencies move with inexcusable slowness and nothing ever gets done on time. The core issue is of course antiquated business processes that are mostly manual and error prone. Any business process re-engineering consultant will be able to improve the efficiency of virtually any government process through some Extreme Konsulting but the process will take too long before we can see any results. Instead, let us begin with a quick-and-dirty solution : Let us abolish holidays in all government offices ! No, I am not suggesting that government employees work round the clock -- the employees will get all their leave but government offices will be open 365 days a year on two shifts from 6:00AM till 2:00PM and from 2:00 PM to 10:00 PM as is the case in any manufacturing or BPO sector or even in the railways !

Working extra shifts will not need extra people -- our sarkari offices are in any case over staffed and bloated with all kinds of people who sit and do virtually nothing throughout the day. Since they cannot be sacked, laid off or retrenched, let us make them work extra hours to clear papers as I have explained in my post Who needs holidays ?

The first area of government where this should be tried out is the courts, where we all know that there is a huge backlog of cases which will, in the normal course of events, take hundreds of years to clear. A quick look at our judicial system would show that most court infrastructure -- especially the lower and district level courts -- are grossly underutilized and a two shift approach can easily double the throughput. We could make things move along even faster by introducing clever business processes that limit the time to each litigant and make sure that there is no corruption in the fixing of dates so that the time it takes to resolve a litigation can be significantly reduced. What is possible with courts can then be extended to all other departments of the government.



We in India, as the proud descendants of the Sindhu Saraswati civilisation that was first discovered at Harappa and Mohenjodaro and which flourished on the banks of the now lost Saraswati river, have the duty and responsibility to ensure that this nation regains the capability to lead the world again in thought and deed. To do so we need education, we need to stamp out corruption and we need to make the government move faster. That is the agenda for India in 2014.

April 06, 2014

Time as a factor in Indian Jurisprudence

image borrowed from
http://integral-options.blogspot.com
The Indian judiciary in general and the judges of the Supreme Court in particular have been in the news for taking some spectacular action in certain high profile cases like Sahara / Subrata Roy, Srinivasan / IPL, 2G / CWG scams and now of late, after the Nirbhaya episode, in rape cases. However, for the vast majority of Indian citizens, the Ram, Shyam, Jadu, Madhu who sit outside numerous high courts and district courts sipping tea and waiting for their cases to be heard and disposed of, the great terror is delay ! In India, litigation never ends -- it goes on and on and on until litigants die or use money and muscle power to settle out of courts. The complex series of adjournments, non-availability of dates and legal skulduggery ensures that cases never end and the Indian judicial system must be the poster boy for the maxim that justice delayed is justice denied.

All this is well known, what is not known is what can we do about this. In earlier posts I have tried to articulate some ideas so let me recapitulate the same and then propose something new.

1. First and foremost, we need more judges and magistrates at all levels. Like all government agencies in India the judicial system is also overstaffed but it is overstaffed with the useless clerks, peons and other support staff, not actual judges. Given the large number of qualified lawyers floating around our courts, appointing new judges would be easy -- what is impossible is to find rooms where they will hold court. This is where I suggest, as I have done in my post Sweating Assets to Expedite Justice, that we operate the judicial system in three-shifts, in a 24 x 7 mode. This will double if not triple the throughput of judicial services in the country and be the first step.

2. While lawyers and some litigants have a vested interest in delaying cases, it should be the judge or the magistrate who should have, and own, the responsibility of expediting the delivery of judicial services. We in the corporate sector are used to the concept of Key Performance Indicators that are decided on by management and which have to be met by executives before they are promoted or otherwise incentivised. Such KPI's are of course alien to government employees like the judiciary but it is high time that such ideas should be implemented. In my earlier post on Measuring & Monitoring Judicial Efficiency, I have proposed a set of 4 simple metrics through which a judge can be measured on both (a) the speed and (b) quality of his work. These metrics may be debated, discussed and refined but we must accept the inevitability of measuring and monitoring judicial efficiency. This must be tied to appropriate rewards.

3. Third is the concept of time -- something that does not seem to exist in the minds of a judge in India. The simple assumption is a case must take as long a time as is required to dispose of and there is no upper limit on anything. This is perhaps the most dangerous concept so let me explain this greater detail.

In principle a litigant's lawyers can go on talking for hours, seek an endless number of adjournments and in general delay things quite a bit. We understand  that judges of late have woken up to this dilatory practices and have tried to curtail this by disallowing adjournments but the effect of all this is perhaps visible in the Supreme Court and some High Court cases but not so in the lower courts where the vast bulk of litigants and their cases are stuck.

In fact, the delays and the resultant disappointments are not simply because of adjournments -- there are other, more nefarious factors playing here. Consider the following :

  • When a case is adjourned or deferred, a next "date" is fixed. This is the first level of corruption. You need to pay a bribe to the clerk to get the next date within a reasonable amount of time..
  • But even if you get a "date", there is no guarantee that your case will be heard. Someone else will also pay money, perhaps more money, to have his or her case scheduled on the same date ahead of  you and the judge is generally under no obligation to hear your case just because you have been given a date. Without contempt, let me state that many, if not most judges in India have an arrogance dating back to feudal times, and it is beneath their dignity to treat litigants as customers of judicial services. So if they feel like it they, simply stop hearing cases on a particular date and walk out, unmindful of the hardship that they are imposing on the litigant waiting patiently in the courtroom.
  • And what is worst is if the litigant's case is not heard on the scheduled date, it is not that it is heard the next day -- it is postponed to another future date which has to be obtained again,  by paying more bribes. This is the what really kills the litigant.
So what we need is at transparent scheduling and time management mechanism in the judicial system. Let us see  how we can go around designing such a system.
  • First like chess players, we need to allocate a fixed amount of time to each litigant to have his case argued. The bench and the bar can sit down and decide upon the quantum of time to be allocated for different kinds of cases -- say civil, criminal, tenancy, income tax, and at different levels like district, High and Supreme courts. Once the time is fixed, there must be clock, just as in a chess game, that will measure the amount of time spent by a litigant and this will form a part of the court records. And like Twitter that simply would not allow a single character after 140, the judge must not allow a single minute more to be spoken or otherwise allotted to a litigant.
  • The entire block of time need not be consumed in one day. It may be spread across three or four days but the cumulative figure must be tracked and adhered to. Obviously this calls for creating a computer system to keep track of the cases being heard and the amount of time consumed in each. This is hardly a challenge for any software developer who are used to tracking hundreds and thousand of items in a retail store or thousands of airplane flights taking off and landing in airports around the world. Such a software can be built easily. What we need is the management ( or judicial ) will to implement it. This calls for education, change-management and finally good big stick to beat people who fail to fall in line.
  • What is important however is the scheduling of cases. When a case is adjourned, there must be an automatic system that will schedule the next hearing based on (a) the number of cases pending before the same judge and (b) the duration of time left for the case. Algorithms to create such schedules are easy to make and there can be a manual override to handle situations where the litigant, the lawyer or the judge has a problem or a schedule conflict.
  • The schedule that shows the cases listed and the time allocated to each case must be a public document that is freely accessible on the web. Schedules can slip because of exigencies and circumstances but the slippage must be reflected immediately on the public website along with all cascaded changes. So if the case for A cannot be heard on day D, then it must be heard on day D+1, not on some indefinite day D+n. If this means that the case for B scheduled for day D+1, cannot be heard then it should slide to day D+2 and not some distant date D+m. Since the modified schedule is in the public domain, B should keep track of his case and note that this case has been postponed from D+1 to D+2 and plan accordingly. If D+2 is a problem for B then he should be allowed seek a change and the scheduling software will create a new schedule not only for B but all others down the line.
  • Since the schedules and their change are always in the public domain, it will always be possible to have audit trails that will allow the senior management ( or the higher judiciary ) to make sure that all schedule changes are on the basis of genuine requirements and any aberration can be quickly spotted. Also litigants will have a clear visibility on the days their case will be heard and will be spared the immense harassment of going to the court and waiting and hoping for their case to be heard.
The technology to make all this happen is easy to design and build. Moreover this concept can be introduced with far less disruption than either (a) three shift courts and (b) metrics for judges because judicial staff will have no reason not to agree with this change. However this transparency will have a severe and crippling impact on the lower judicial from making a lot of money and hence it may be opposed tooth and nail. But if the higher judiciary can make this happen, then litigants in India will be forever grateful to their Lordships.

March 27, 2014

India : A failure of management -- Aadhar

The failure of Nandan Nilekani and the UIDAI to introduce the Aadhar card into the the mainstream of India's economic life is one more example of management failure in India. What is most frustrating in the case of Aadhar was that there was a clear need for Aadhar, patterned on the US Social Security Number, and the biometric technology was very much in place to uniquely identify each "warm body" and connect him or her to a unique number. Yet we failed - Why ? Let us study this in a little detail and then try to generalise the same for other similar projects across the country.

I believe that Aadhar failed because of a combination of one or more of the following reasons :
  1. In general Indians are suspicious of and reluctant to adopt new practices. Starting from social mores like homosexuality and inter-caste marriages to business practices like insisting on revenue stamps and stamp paper for legitimizing contracts or looking for three (possibly dummy) tenders for every purchase requirement -- India is very hesitant to break away from the past.
  2. Aadhar would have made a major dent in the ability of corrupt middlemen, both government employees and private contractors, to siphon of public money before it reached the eventual beneficiary. So there was a caucus or cabal that was bent on ensuring its failure.
  3. Aadhar had the biometric technology to connect each human being to a unique number and it would have been successful if it had restricted itself to that role. However a "scope creep" made Aadhar to record the residential address of each person and this is where UIDAI fell into a trap because
    • UIDAI had no way to verify addresses and had to fall back on the traditional, "address proofs" like Bank passbook, Utility Bills or Passport. So the legitimacy, or accuracy, of the Aadhar card was brought down to the same level as so many other cards -- PAN, Voter Card, Drivers Licence -- that are floating around the country
    • Getting equated to cards like PAN and Voter Card opened up turf issues. In particular, the security establishment, led by the Ministry of Home Affairs, got particularly anxious because the equation of the Voter Card with Aadhar card made it particularly vulnerable to abuse by illegal immigrants in communally sensitive election constituencies. 
  4. The final nail in the coffin of Aadhar was the extreme mismanagement of the Aadhar issuance process. Other than the showcase rollout in Delhi and the major metros, the implementation at the district level towns was a total nightmare. Ineffective communication, incomplete planning, ill-trained staff, badly designed software and inadequate computing and communication infrastructure [ and I say this out of personal experience at the Aadhar camp at IIT Kharagpur, where I failed to get my card ] made sure that (a) most people could not get cards and (b) the credibility of the card was compromised beyond repair.
  5. The arrogance displayed by the UIDAI in denying the reality of the previous point 4 and using its proximity to the PMO to arm-twist the population to fall in line by threatening to withhold critical government services like subsidized gas cylinders without which the citizen cannot cook his daily meal !
Point 5 led to the solidification of public resentment against Aadhar to the point that other, more substantial,  objections like (a) the right to privacy that was being blatantly violated and (b) the immorality of data collected with government funds being handed over to a foreign private body -- that few people had a clear idea about, became red-hot issues that the nation started talking about. The resultant tsunami of public opinion against Aadhar was something that roiled around the country and when it reached the doors of the Supreme Court in the form of a PIL, there was little that the judges could do but agree with the petitioners  and remove all the compulsory features of Aadhar and reduced it to the level of another petty loyalty card from retail chain !

So ends the Aadhar story but in this lies the story of many other failures in India where despite having the technology and the money to afford it, we simply cannot implement a solution. Consider ...
  • Roads - the technology to build roads is ancient and yet why is that our roads are full of craters ?
  • Electricity - again the technology to generate and distribute electricity is well known as are ways to detect and prevent theft. Nevertheless there are large parts of the country that are perpetually short of power.
  • Schools - we have thousands of unemployed graduates who would be glad to teach in schools. Building a school is neither expensive nor difficult and yet we cannot provide schools as a matter of right to our kids
  • this list can go on and on and on ...
The problem is neither with technology nor the money to afford it. The real problem lies in our collective inability to manage contradictions and put all the processes together so that we can deliver a solution that is necessary for our citizens. 

Or in short we simply cannot manage the entire process because as a nation we, at all levels starting from the gram panchayat all the way to Rashtrapati Bhavan, lack the basic managerial skills that are necessary for a country to progress and prosper.

So, what is the solution ? Should the Government of India open a 100 more IIMs in the country ? Unfortunately if the IIMs had any clue about how to really manage anything India would have been very different today. The answer is far more complex and will be explored in a subsequent post.

February 26, 2014

Data Science - a DIY approach

Everybody wants to become a data scientist but given the huge number of tools and techniques that are available no one really seems to know where to start and how to go about acquiring the right skills.
image taken from http://www.rosebt.com

In the process of teaching data science to students at the Praxis Business School I have realised that there is a huge amount of information that is freely available on the web and thanks to Google, one can find this information very easily. If one is motivated enough, one can learn data science on his own. In an earlier post I have given a very broad Introduction to Data Science but in this post, that I plan to update every now and then, I have tried to assemble a collection of tutorials, grouped by topic that will help anyone get a good grip on this subject by actually doing something practical.

R : Statistical Programming Language

Forget about SAS, SPSS and other proprietary languages that want to extort money from you and simply invest your time and effort in mastering R, a free but excellent tool that is used most widely in data science.

Here is a "Level 0 tutorial for getting started with R" by April Galyardt that will get you started. This tutorial will tell you how to install R but you should also consider installing R Studio, a free graphic development environment that makes R easier to use.

Galyardt's tutorial is very comprehensive but if you are that impatient type for whom doing anything in depth is too tiring and yet you want to be familiar with this cutting edge technology then I suggest that you go through this 5-part Beginners Guide to R from Computerworld.

Both these two tutorials require you to install R on your machine, which is not too difficult but then even if that is too much for you, I suggest that you visit Data Camp, or Try R at the O'Reilly Code School. This will give you a flavour of what is it that people do with R.

Statistics

The ancient science of statistics has suddenly got a new lease of life and celebrity status because of a sudden surge of interest in data science. If you leave aside the Map Reduce and the associated world of Hadoop, then [ statistics + statistical tools ] is virtually synonymous with data science. Hence it is essential to get a good grip of statistics.

As an ancient science that is taught in many undergraduate and postgraduate courses, there are many excellent books on statistics but today's data scientist must not only be proficient in statistics but also in a tool like R.

An excellent place to start learning Statistics along with R is the open source, copyright free OpenIntro website. Not only will this allow you to download an excellent  book in PDF format but will offer you a set of lab exercises based on R where you can try out what you have learnt.

Other fairly comprehensive tutorials on statistics with R are available from Kelly Black of Clarkson University. and from King of Coastal Carolina University. However if you are seriously interested in a career with data science you should purchase a regular text book on Statistics for reference purposes and for this you can look at either Statistics for Management by Levin or Applied Business Statistics by Black.

Traditional statistics is generally restricted to descriptive and interpretive statistics that is well covered in these links. However data science is more interested in predictive statistics and this is addressed in areas like Data Mining and Machine Learning that will be addressed later. However check out this book for a quick Introduction to Data Science  and also download a PDF file of the book.

For a look at using R for more advanced tasks you can take a look at this "meta book" that points you to other books that shows you how to tackle more complex problems.

Python with Anaconda

Python is another very strong challenger for the position of the best, or most powerful tool, for data science and it makes sense to get a hang of it. This is because it not only supports much of the functionality that R provides, through the Pandas library, but is also quite compatible with Hadoop and the world of Map-Reduce.

A quick way to get a hang of Python without installing it on your machine is to try out the Python tutorial at AfterHoursProgramming or somewhere similar. But  a better way to get going would be to download Anaconda, that not only installs Python but also provides a rich GUI interface to execute Python programs in addition to the standard command line approach. Once you have your Python (with or without Anaconda) in place, you can try out this workshop from Open Tech School free of cost. Or you can jump straight into this very comprehensive tutorial on using Python for data science (alternate )that actually uses data from a Kaggle competition.

Visualisation

Visualisation is another key area of data science because a business wants data scientists to tell a good story with the data. My blog on visualisation is a nice starting point for all those who are interested in getting a quick hang of to get started with free tools like Google Charts. Tableau is an excellent and widely used tool that is used for visualisation that has a  public version for free download that you can try out with the free tutorials available in my blog post. Other excellent tools are Google Fusion Tables that you can explore here or Chartbuilder that you can try out online here or learn more about in this tutorial.


Data Mining / Machine Learning

As we said earlier, traditional statistics is generally limited to descriptive and interpretive work but the world wants predictions and predictive statistics is where we get into data mining and machine learning. Please understand that data mining and machine learning is a complex subject and you need to get a good grounding on the algorithms that are used for Classification, Clustering, Association Rules, Text Analytics and other complex tasks. A quick overview of all these techniques is available in this Overview of Data Mining Techniques.

It is difficult to take a short cut through this subject but if you have a basic idea of what all this means then you can try out the examples and exercises given in this book on R-DataMining. Rattle is an excellent add-on to R that gives you a GUI interface for Data Mining. You can download rattle and then try out with this short but descriptive tutorial with this data.

On the other hand if you are more of a programmer and less of a statistician, you may prefer to use Python for your task. A good way to get started is to download and read "A Programmers Guide to Data Mining" that not only introduces the subject but also gives loads of ready made Python code for you to try out.

Evergreen SQL

Last but not the least, relational databases and SQL is an absolute must for anyone who is interested in data science but since it is so well known and so widely used in the technology community that I refrain from singing its praises or giving any pointers to how it can be learnt.

If you go all through all this then you are almost there on your way to become a data scientist. The only thing missing is Map-Reduce and Hadoop that I will write about only when I personally know enough about that subject. In the meantime subscribe to the twitter feed of KDNuggets to keep yourself abreast of what is happening in the world of Data Scientists -- that the wise men at Harvard have declared to be the Sexiest Job of the Twenty First Century.

This survey has addressed the tools and techniques that a data scientist uses in his daily job. What is missing here is an understanding of the business domain -- like Retail, Finance, Telecom -- where these tools need to be used to deliver business value. To learn all this you may consider joining the One Year program on Business Analytics at the Praxis Business School, Kolkata  where we teach all this and more. Sorry for that little advertisement but I need to earn a livelihood as well :-)


I will be updating this post based on changes in my knowledge and perception of this field, so you might see some changes every now and then.   Last update 28 Mar 2014, 04 Apr 2014.

January 23, 2014

Fighting Corruption without Anarchy : Big Data Analytics and RTI-2

Ever since Mohandas G dreamt up the concept of civil disobedience, Indian politicians believe that bandhs, dharnas, rail-rokos, chakka-jams and other anarchic activities are the only tools available to do anything for the welfare of the people. Even educated people like Kejriwal and Jyoti Basu (may he NEVER rest in peace for the damage that he did to West Bengal ) have taken the easy way of out by creating a ruckus on the road to get their point of view across.

Let us see if there is a sane way to fight corruption in this country.

We all know that the basic rules of checks and balances are all there. Approvals, authorizations and the inevitable CAG audit followed by a CBI investigation should have been enough to stop corruption but when the "policeman is himself the thief" then all these checks and balances go right out of the window and are of no use anymore. So what else can be done ?

"Sunlight is the best disinfectant," is a well known quote from the US Supreme Court Justice Louis Brandeis that refers to the benefits of openness and transparency in public life and the RTI Act, passed by the much maligned Congress party in 2005, could have been a game changer in this respect. Under this Act, any citizen of India can request information from a public body and ideally this could / should have been used to scrutinise and sanitize public finances. Unfortunately, the law has been used, abused and misused by both requesters and suppliers of information in a variety of ways so that its efficacy has been largely blunted. While I do not wish to go into the details of the actual lacunae, there are two areas where the abuse has been quite significant. First, people use RTI to ask for junk information or for information that will help them in personal or corporate financial deals. Secondly the information is very often given out by the government agency as reams of paper that cannot be examined to any great detail, nor corroborated or compared with other similar information. So no meaningful analysis is possible.

In an earlier post on how RTI data can used to curb corruption I had tried to describe a mechanism by which every government agency would be mandated to publish its audited financial data on the internet in XBRL format without having to be told to do so under the RTI Act. Then we could have had computer programs ( or "bots") study the reports automatically and point out deviations and discrepancies which the government would have to explain under the formal RTI process.

Today, I am thinking of and advocating something even simpler.

At any point where there is a movement of money from private to public ( or government ) hand there must be some kind of a challan or document generated. Similarly, whenever there is a payment of public money to a private person or company there must be a similar document being generated and signed by someone. In either case, this document is available inside some government premise and in the custody of some government official -- who can be employee of the central, state, municipal, panchayat or some other government body.

Let us demand that the contents of each and every such document, without exception,  be compulsorily made available on the public internet within 7 days of the money changing hands. This means that we will have a public record of all, repeat ALL, movements of public money both inward and outward from the government coffers. However in the interest of privacy and security we could sanitize the data by removing identities of certain payees ( like Income tax payments, refunds ) or block data on sensitive matters like strategic items in defence ( though not the purchase of soap and socks for jawans ). However the vast number of payments that the government makes to employees, vendors, contractors, NGOs etc must compulsorily be made available -- without the need for a formal RTI demand.

Such a project while vast in scope is no more difficult than building the Golden Quadrilateral under NHAI, the UIDAI project or even the Pulse Polio program. In fact since it is only dealing with data lying  inside government premises, and in the custody of a government employee, it can be done far more easily and that to in a phased manner.

Once we have the data, now what ?

Today we have powerful tools for handling Big Data -- Hadoop, Map Reduce and many others -- with which it should be quite simple to create a complete trail of where money is entering the system, how it is flowing across the system and how it is being finally disbursed. For this, there is no need for the government to put in any formal effort because many well meaning RTI activists can use these data analytics tools to locate and pinpoint anomalies and discrepancies in the movement of money and then focus on these areas with more specific RTI queries.

this lovely image of the flow of Big Data is taken from "New Value Comes from Analyzing Diverse Data"

Initially, the numbers may not add up as payments may not equal receipts but then the government must explain the gap in terms of known economic tools like deficit financing, market borrowing and others. Again this could be an ongoing process with the gaps being closed with the passage of time.

Bribes will of course flow outside the system. Which means that money changing hands in 2G or CWG scams will not be trackable here. But what can be spotted is the effect of the  bribes ! When abnormally high payments are made for commonplace items, when thousands of fake recipients get NREGA money, when too much money is being suddenly funneled through specific individuals or offices then deviations from accepted norms will happen and standard data anlytics tools can be used to spot these needles in the vast haystack of government finances.

But that will only happen if government finances are compulsorily made available to the public. That should be the goal of RTI-2

December 10, 2013

Currency, Commodity and Bitcoins

December 02, 2013

Bitcoins : As I understand it !

In an earlier post, "First Look at a New Currency",  I had written about my hesitant entrance into the world of the Bitcoin economy and today, the Economic Times has an article that shows that even the regulatory bodies in India have started to have a point of view on this strange and disruptive technology. Since then, many people have been asking me about this piece of technology so let me take a stab at trying to understanding this currency that has appreciated from just US$ 200 to US$1000 in the space of just one month.

First question : What has caused this sudden jump in the value of Bitcoin ? Well the answer is that US Regulatory agencies have declared that there is nothing inherently wrong or illegal with the usage of this currency and the US cannot fall behind the rest of the world, and particularly China, in adopting this new technology. This has set off a flurry of trading activities that has resulted in this huge increase in the price. Second question : Is this a bubble ? Perhaps it is too early to call but let me explain how this works and then you can take your own decision.


Traditional currencies are printed, or created, by a government monetary authority like the RBI as per its discretion and convenience and the value of an individual unit of currency ( one Rupee ) is set by the free market. If many people want to buy Rupees and are willing to pay in US Dollars then the value of the Rupee goes up and you need to pay more Dollars to get a Rupee. If the value of the Rupee goes up then you also need fewer Rupees to buy a barrel of oil or a kilo of rice. On the other hand if people want to get rid of Rupees and buy US Dollars, then the price of Rupees fall and you need to pay fewer US Dollars to buy a Rupee. As a corollary, you would need more Rupees to buy a barrel of oil or a kilo of rice. To allow the man on the street to buy a kilo of rice, the RBI has the liberty (and the legal authority ) to print more Rupees and hand it out as a dole to Indian citizens ( the NREGA scheme and similar stuff) but with more Rupees in circulation, it's price goes down anyway and the currency depreciates. So net-net, the value of the currency is a function of its demand-supply in the market PLUS the discretionary power of the central monetary authority to print and release ( or withdraw ) units of the currency from the market.

This is how all national currencies operate until Bitcoins arrived on the scene and the currency market may never be the same again ..

Actually Bitcoins are less like currency notes and more like a commodity. Say gold.

The total quantity of gold in the world is fixed. No government can create or destroy gold and its price is set by market forces and the constraints imposed by demand and supply. However you can set up an elaborate mining operation and get the gold out of the ground and this is the only way that the supply can be increased. Moreover there is a fixed amount of gold that is available on the planet -- though we may not know exactly what that amount is.

Just as gold can, technically, be mined by anyone with a shovel and access to a piece of land, a Bitcoin can be created, or "mined" by anyone with a computer running a specific software and solving a class of mathematical problems. However these are not simple mathematical problems but extremely tough problems -- similar to finding the next prime number or finding the prime factors of a large number -- that consume a huge amount of computing resources. If you were to try solving these problems on your laptop it will take thousands of years and you may still not be able to solve it. Just as you can dig in your garden for years and never find the gold.

But if  you do solve the problem ( or do find the gold ) and the rest of the community accepts that your solution is correct ( or the rest of world agrees that the rock you have found in your garden is indeed gold) then you have "mined" a Bitcoin and you can call it your own and put it into your "wallet".  I will explain that "wallet" part in a minute. Do note however that the Bitcoin that  you have mined has not been created or printed by RBI or any national monetary authority.

However if you do not wish to solve the mathematical problem ( or go digging in your garden ) you can still get a Bitcoin ( or gold ) by going to a someone who has a Bitcoin ( a jeweller who has gold bar ) and buy a Bitcoin using any other convertible currency that you have.

So you either a mine a coin or buy one. Mining a coin ( like gold ) is difficult and so most of us leave it to the specialists and professionals. Instead, we can and do buy Bitcoins from the market.

But unlike a physical piece of gold, a digital artifact can be easily reproduced and used by multiple people. You can buy a copy of a book as PDF file or music in an MPG format and then you can make multiple copies that you can  give ( or sell )  to many people. If that were to happen in the case of Bitcoins then the economy will collapse ( as publishers of books and music are realising when they see their assets being copied and distributed at low or no cost ).

This is where the concept of a "wallet" comes in.

A wallet is a piece of software -- think of it as a password protected spreadsheet -- that has in it the data of how many Bitcoins the wallet contains at the moment. Each wallet is identified with a unique number -- but not the name or identity of the owner -- and when a Bitcoin moves from one wallet to another, the quantity of Bitcoins goes up in one and goes down in the other. This is nothing new, two entries in a ledger, one credit and one debit.

But why can I not fake this ? I send one Bitcoin to someone, so he gets a credit but I do not not debit my ledger and continue with my original balance. This is where the magic of the Bitcoin "network" steps in by ensuring that every wallet on the planet ( or Bitcoin network ) is synchronised with every other wallet on the planet. Since every wallet has a copy of the ledgers lying in all other wallets it is impossible for any one person to make a fake entry in his wallet. Even if two or three people collude, that will not work because all other wallets will not be in sync with the culprit(s) and the culprit(s) transactions will not be confirmed by other wallets.

Only if more than 50% of all wallets in the world can be simultaneously compromised with fraudulent data then, and only then, will it possible for someone to make a fraudulent entry into one's own ledger and make a double payment with the same Bitcoin. The probability of this happening is very low and is statistically negligible. So this shared and syncrhonised wallet ensures that even though a Bitcoin is non-physical artifact, it cannot be used more than once by the same person to make a payment. So it is like a real physical coin that can be spent only once.

So the Bitcoin currency is defined by (a) mathematical problems that, on being solved, create a bitcoin and (b) wallets that store the history of every transaction in the form of a continuously updated "blockchain" information.

And finally, here are some more pieces of information that might explain the economy better.

a) The mathematical problems that define the creation of Bitcoins are such that the total number of Bitcoins will be limited to about 21 million and the production will slow down and stop around 2050 CE when it will become almost impossible to solve the mathematical problems. It is as if the world will run out of gold or oil.
b) Wallets can be kept in your personal laptop or smartphone but it may be better to keep it with a web based service provider. This is like saying that you can keep data as an Excel spreadsheet in your laptop or you can keep the same data as a Google Doc spreadsheet in the cloud and can access it with your Gmail ID. In banking terms, this would mean that you leave your wallet in a safe-deposit vault with a bank.
c) You may also choose to transfer your Bitcoins from your wallet to an account with a bank or currency exchange. This is analogous to depositing cash in a bank, or converting your equity shares from a  physical form  to a demat form and place them with a depository. It could also be something like depositing gold with a bank and getting a receipt for the "demat" gold.

After playing around with creating wallets, acquiring Bitcoins and placing them in my wallet -- as described in my earlier post "A first look at a new currency"  -- I have recently opened an account with MtGox, the world's best known Bitcoin exchange so that I can sell my Bitcoins, if I want to, and get the money into my StanChart Bank account in India. For this I had to provide the full KYC information including identity proof ( passport ) and address proof ( phone bill ). This ensures that Bitcoins cannot be used for money laundering or any other illegal activity.

This in short is Bitcoin. Please note that this area is so new and evolving that my personal understanding is changing almost every day and this post may be updated or altered with more information. If you can explain this better, please do so in a comment and I will be happy to learn from you.
--------------------------------------------------------------------------------
Update : Mt. Gox was robbed of all its Bitcoins through a very sophisticated cyber-heist and has filed for bankruptcy. Fortunately, I had not transferred my coins from my personal wallet to Mt.Gox. Hence, they are safe, so far !

About This Blog

  © Blogger template 'External' by Ourblogtemplates.com 2008

Back to TOP