Saturday, October 24, 2015

Book Review: I Am Malala

I Am Malala: The Girl Who Stood Up for Education and Was Shot by the TalibanI Am Malala: The Girl Who Stood Up for Education and Was Shot by the Taliban by Malala Yousafzai
My rating: 4 of 5 stars

For a 15 year old to have such insight, drive and the grit, it is astonishing to see Malala just going ahead with her business and not give up. Having faced such hardships right from the environment in Swat valley to being shot, Malala is a symbol of courage and I have garnered a lot respect for this nobel prize laureate.

Regarding the book, I started reading this book to gain some real insight in that part of the world and I am glad I started with this masterpiece. The book covers a bit of history of Pakistan to begin with. Then Malala goes on to talk of her childhood friends and family. Her grip on penning down what she feels of her parents and how they look at it is again commendable. Its quite inspiring to see Malala and her father fighting for the fundamental human right of girls education knowing there is a threat looming all the time. You see how both of them are not afraid come what may but when it comes to the other, they do get protective. Amidst all the life story, Malala presents an amazing account of a father-daughter relationship. Malala seems to have a lot of respect for her culture and religion at the same time she is not willing to be fooled by the impositions of gender inequality. Reading this book is a reminder of how much we take for granted things around us. I am sure I will appreciate my freedom a tad bit more now. I hope and wish her dream of getting back to Swat valley and the vision she carries of her motherland does come true. The world needs more Malalas. Looking forward to reading more about her life as she carries on the fight she has taken.

View all my reviews

Wednesday, October 21, 2015

Monitoring Java based Applications: Metrics+Graphite+Grafana

As developers when we write a piece of code we try our best to make sure we have carefully chosen the best algorithms, the data structures are carefully tuned to match the situation at hand and the piece of software we wrote just works out of the box. Unfortunately life is not ideal and there are some hiccups that are bound to hit us however smooth the ride be. Some nasty bugs will inevitably show up on the production server as the software is used for a length of time. The best one can do then is to monitor the software being used in production environment, anticipate the problems before it hits the user and hampers their experience.

This post will talk about such a stack for monitoring java based applications involving:

We will use dropwizard metrics to instrument our Java code and push the data to graphite. Graphite will store our time-series data meaning values such as latency, performance, throughput, counts etc corresponding to say our APIs. Though Graphite renders graphs built on top of the time-series data as and when required, Graphite on its own doesn't provide the best visual experience and dashboard properties. We will thence use Grafana to build and view our dashboards. So the pipeline on a high level will be this:



Here is the documentation of dropwizard metrics listing its various capabilities. Their documentation is not that great, in my opinion, but workable. The good news is there is also a spring library that provides annotations you can use directly to enable instrumentation. Just add an annotation on top of a method and thats it. Here is an example:

@Metered(absolute = true, name= "RuntimeError")
public ErrorMessage handleThrowable(HttpServletRequest request, Throwable t) {

            ………..

            }

As you see here, there is an annotation (@Metred) which is responsible for instrumenting the following method- handleThrowable(...). The metred annotation gives the timed average meaning number of errors in a unit of time. As simple as that! The code in bold is all you need to write to instrument the following method, for instance.

Next, we need to tell our metrics where Graphite resides so data can be sent to Graphite. Here is how you configure this. The prefixedWith() attribute defines how Graphite will show this. Here is a screenshot from my Graphite dashboard showing the tree-based structure that we just set.


graphite-left.png

On the right of this pane shown above you ca see the graph you select. The visual features in Graphite is rather limited so we will forward the Graphit input to Grafana. Here is a video introducing Grafana interface.e



Make your dashboards the way you wish to see your graphs and keep monitoring to stay ahead of the bugs!

Tuesday, October 20, 2015

Log Archiving with Logstash

A crash, a panic attack and we rush to the logs hoping it comes to rescue. Alas there is so much content in these logs that finding the relevant content itself is a herculean task. It is thus a good idea to think upfront and brace yourself to handle the panic situation better. Keep all the logs ( application logs, database logs etc) nicely organized in a tree structure all of them being kept at one place. More technically, it is a good idea to archive your logs and have a way to make sense of the information hidden in logs. I was recently faced with such a situation and ended up making an ELK based stack.


I started with syslog-ng. Syslog has been around for sometime so it gave me a sense of stability and good community around it. Unfortunately syslog is a bit outdated too! The next thing we considered was logstash. Logstash along with Elasticsearch  and Kibana forms a powerful stack, popularly known as the ELK stack.


For this blog post, we will limit our scope and stick to logstash part of the ELK stack. We will make an archive which collects logs from different web and database servers and archives them all in one place. The intelligence and visual analysis are left to Elasticsearch and Kibana respectively and thence not part of this post. Following is a very high level architecture I came up with:


archiving architecture.jpg
We are going to work with two pieces of software here:
  1. logstash-forwarder
  2. logstash
We will install logstash-forwarder on the clients (our applications that generate logs). This is an agent that sends the logs to a central archive server. Note that logstash-forwarder needs to be installed at EVERY client. We will next install logstash on the archiving server. This will receive logs from every client and archive it in a format that you can define. Here is what I came up with for my archive structure.
archive-structure.jpg


I like to archive by time and then the log source as depicted in the picture above. Now that we are done with sorting the architecture and our logging format we need to next configure our clients and the server. Digitalocean has done a wonderful job of explaining the installations and configurations so I will leave you with this article. You can skip the parts involving Elasticsearch, Kibana and ngnix.

Once you are done with archiving make sure you have a way to periodically clean logs from the application servers, if at all you store last 1-2 days of logs there too. For this I set up a cron job that cleans up logs older than a week in my case.

Tuesday, August 18, 2015

Book Review: I Too Had a Dream

I Too Had a DreamI Too Had a Dream by Verghese Kurien
My rating: 5 of 5 stars

With autobiographies I sometimes don't understand how deeply you trust what you read, how you verify the facts and what alterate sources to trust. Thence I am generally not a big fan of reading autobiographies even though I love biographies. Nevertheless I am happy I picked up this book for this is an honest account on Mr Verghese Kurien. You can feel the empathy for customers in his words. You can see his intent and passion for cooperatives.

The book begins with early life of Mr Kurien and how through an accidental turn of events Mr Kurien was forced into diary engineering and then working in Kiara, how Sardar Vallabhbhai Patel and Tribhuvandas affect his life and vision and how he fought through all the politics and bureaucracies to make one of the biggest cooperatives in India. His constant emphasis on cooperative vs corporate is a food for thought. Though he blatantly criticises corporate and sometimes you tend to disagree with him this book presents a good view of the two worlds- cooperative and corporate Further, the obsession of author for the delicate balance between the benefits of both producers (farmers) and consumers of milk (or other agricultural products) is very compelling and provides for an engaging lession on marketing.

Amul and Dhara as brands stand as tall success to prove the success he achieved. The real success of Mr Kurien is way above these though. This book thus classifies as a must read for the impeccable writing plus insights on cooperative strategy and marketing.

View all my reviews

Monday, August 10, 2015

MobileSpark conference 2015: Impressions on Day 1

MobileSpark is a two day conference and MobileSpark 2015 happened last Friday and Saturday (August 6 and 7, 2015). I attended the complete event and here I put down some impressions from the event. I tried to lure some of my friends to join me, also thinking of the hackathon but no one did. Perhaps the agenda did not look as compelling to them. Fair enough, I thought if I am interested I should simply go. The venue was far, around 20 kilometers from my house but I did manage to attend both the days and all the events. Lets get on with the nitty-gritty details now with the two days.

The event was due to begin at 9:30 in the morning, it didn't. Until 10:00 there was no sign of the first talk. Then appeared an over-enthusiastic and loud anchor. The first couple of sessions were not very engaging enough to change the perception either. My enthusiasm dropped a bit. I started to feel my friends who decided against attending this conference were right. I was going to find it a bit tough to spend two days here. Thankfully it did pick up and I sort of enjoyed being there at the end of it all, specially the second day. Lets iterate through the sessions one by one.
  1. The first session was by Ashish Agrawal, CTO, Micromax & YU. He began crunching some numbers to show that Yureka OS had already made some fans and further went to essay the importance of community contributions in building their suite of software. Ashish thanked them and invited more contributions.
  2. Next was a talk by Arvinder Gujral, Director, BD, Twitter APAC. He talked of Twitter and the apponomics of today essaying how startups can leverage twitters's SDKs (Fabric, Crashlytics etc) and user-base to gain traction.
  3. Next, it was Varun Khurana, CTO, Grofers talking of e-commerces of today namely, Flipkarts and Snapdeals and how hyperlocal was the new thing on the block. Indeed there are a ton of hyperlocals and even those that only aim to facilitate hyperlocal startups by facilitating parts of the business. Grofers had a big presence on this conference in general and even stood out as one of the sponsors of the event.
  4. Next, we had Rajat Upadhyaya, VP, Engineering Urban Ladder talking of the space Urban Ladder is in and why the multiple-app strategy they have built makes sense. In their case they have three customer facing apps.
  5. Following this was the best presentation I have come across in some time. Girish Mathrubootham, Founder & CEO, Freshdesk came across to talked on how to build a global product and how to assemble a rockstar team for the same. Girish's was surely the best presentation while he talked about how Freshdesk began and his philosophies on the hierarchy structure, recruiting, managing his employees etc. It was quite thought provoking. As Girish finished his session he got an standing ovation from the crowd and deservedly so.
  6. Lately I have become a fan of the culture InMobi has and I was really looking forward to the next talk but there was not much I could collect from this session. The coffee breaks were discarded in respect of time and perhaps had their obvious impacts on me too. After all, we had Chaipoint serving 24*7 their awesome Chai.
  7. Going forward to the vision of OYO rooms. They target the tier-2, 3 hotels and have a really compelling vision to facilitate the hotel experience. Starting from helping both sides through technology to being empathetic about the infrastructure and maintenance of these hotels to the on-boarding and staying experience for guests they have a grand vision and I really do wish they fulfill it. There is a clear space and I am sure this will simplify the lives of a lot of people.
I skipped through the next couple of sessions to grab some food and have a look at all the stalls various start-ups had set up. Here is a list: Hyperverge, Practo, Chaipoint, Yu, Grofers, Freshdesk, Flipkart, Oyo rooms, Process9, Affle, Urban ladder, Freecharge, Appvigil, Jobspire, Street smart, Babajobs, Playo, Makemyroti, Tapwisdom, Klever kid, Notifie, Reverie, Collection point, Msg 91, Mobignosis, Bizenabler, Localoye, Qykapp, Giftxoxo, Driveu, Unbxd, Furlenco. 

In a later post, I will capture some ideas from this list that caught my eye. For now lets move on to the afternoon session consisting mostly of panel discussions. There were three of them:
  1. Mobile & local commerce: your neighbourhood at your fingertips - Varun Khurana, Grofers; Aditya Rao, LocalOye; Navneet Singh, Peppertap; Samar Singla, Jugnoo

    We have talked of Grofers earlier. Basically it is a hyperlocal grocery delivery startup. You can order groceries on their mobile app and get it delivered like BigBasket, peppertap etc.
    There are so many startup delivering products at the doorstep, not so many that deliver service. LocalOye is one such business. They claim to have a big checklist to get a service provider (for example- a carpenter, plumber etc) on board and they manage the whole experience of such a person coming to your house and delivering the service with an acceptable customer satisfaction. I find it a bit tough to agree that they can control their service guys but I will be really happy to be proved wrong.
    Peppertap is another hyperlocal grocery delivering startup and I really don't know yet how their business is different from Grofers but they do operate in multiple cities and promise to deliver within 2 hours.
    Jugnoo is an on demand auto rickshaw app connecting passengers wishing to travel from one place to another within a particular city  and strives towards making Auto- Rickshaws more affordable and user friendly. Being in Bangalore its almost un-imaginable that will ever happen. Samar also mentioned that they could become a partner to the other hyperlocal businesses such as Grofer, Peppertap etc. Jugnoo has also expanded lately jugnoo meals and jugnoo fatafat.

    The discussion revolved around the future of hyperlocal businesses, how big the market is, how many of them will survive in the years to come etc.
  2. Foodtech: delivering all parts of the equations - Shashank S, Dazo; Harsha T, MasalaBox; Karthik V, MomoE; Arpit D, Roadrunnr; Aruj G, Bhukkad

    Foodtech is another buzz word today and one of the most obvious market along with hyperlocal. Dazo aims at making it easy to select food and offers variety too. They have a curated list of a small number of items on the menu everyday so people people don't fret over choosing what to eat.
    MasalaBox has been serving home made (sort of) food in Kochi for roughly six months and they recently moved to Bangalore too.
    MomoE simplifies the payments and Bhukkad prides at being the one that serves healthy food along with varierty and good taste.

    There were issues around packaging discussed and everyone on the panel agreed that packaging is a big deal. They do want to pack completely non-plastic but they don't see a practical, affordable and easy to adopt alternative. MasalaBox has tried being on that front but failed to scale and maintain the packaging standards. So this space is up for disruption.
  3. The leaky bucket : Balancing Growth and Retention

    To me this came as the most engaging session. The idea was that number of downloads to judge the success of a mobile app is a wrong metric. One needs to see how many of them are repeat customers, how many of them uninstalled the app before or after first use, if one came back to install it after an uninstall etc. The panel discussed some common reasons for uninstalls and how to attend to such issues, how to increase engagement on the app etc.

Tuesday, August 4, 2015

Book Review: The Accidental Prime Minister : The Making and Unmaking of Manmohan Singh

The Accidental Prime Minister : The Making and Unmaking of Manmohan SinghThe Accidental Prime Minister : The Making and Unmaking of Manmohan Singh by Sanjaya Baru
My rating: 5 of 5 stars

The review for this book can be divided in two parts. One, the authors presentation which deserves full marks. The author seems in complete freedom to express his views. The biggest worry for a book of this kind is that the author could be curbed by the high profile officials, which doesn't seem to be the case here. The books gives a balanced view on the two terms served by Dr Manmohan Singh. This is a must read for someone who has the slightest interest in Indian politics. For others too it contains a good account on the leadership, politics in general, the importance of image building etc.

Moving to the second part of review, if you have ever criticized Manmohan, you must read this book. The author allows the reader to form his own opinion quoting instances from his experience. And in my view Manmohan is an idol. Yes, he did loose out on his image in his second term quite miserably and deserves some flak for that too. But he deserves an immense respect for his integrity, patriotism, selflessness and for just being who he really was. As you read the book you feel for him. You wish he had some more power, you wish he had established his authority a bit more and lastly you wish he was on the right side. To end with I wouldn't call Dr. Singh the accidental prime minister but a great intellect just fallen in the wrong place at the wrong time. India will do well to home more people like him in politics.

View all my reviews

Wednesday, July 29, 2015

Book Review: Zero to One: Notes on Startups, or How to Build the Future

Zero to One: Notes on Startups, or How to Build the FutureZero to One: Notes on Startups, or How to Build the Future by Peter Thiel
My rating: 3 of 5 stars

The book is good in patches. To start with, the book did not interest me. I somehow continued and then started finding portions which were really engaging. You see the occasional brilliance of Peter Thiel as a really experienced entrepreneur. He comes across with some really valuable insights. Unfortunately, he wavers easily too. I kept trying to guess who the target audience of this book is. Perhaps the author tries to put in something for everyone which obviously backfires. Because this book is not very engaging I wouldn't recommend it to someone who is not very enthusiastic about the topic but I really look forward to reading a masterpiece from Peter Thiel. And I wish the next book from him is better directed to a specific audience in mind.

View all my reviews