Trends in Big Data, Hadoop, Business Intelligence, Analytics and Dashboards

How has the interest in Big Data, Hadoop, Business Intelligence, Analytics and Dashboards changed over the years?

One easy way to gauge the interest is to measure how much news is generated for the related term and Google Trends allows you do that very easily.

After plugging all of the above terms in Google trends and further analysis leads to the following visualizations.

Aggregating the results by year

Image

 

It is very amazing to see that the stream representing Dashboards has remained constant through out the years.

So does the stream for Analytics and Business Intelligence in general exihibit similar trend.

Analytics is kind of widening its mouth as we move forward and that is being helped by the combination of terms such as Hadoop + Big Data + Analytics being used almost together.

Now check the line chart below

Image

 

Looks like the Trend for Dashboards define the lower bound and the trend for Business Intelligence define the upper bound. The trend for Hadoop started around first Quarter of 2007. The trend for Big Data started around third Quarter of 2008 and ever since they both are rapidly increasing. It remains to see whether they will cross “Business Intelligence” in terms of popularity of kind of merge and find a stable position somewhere in the middle.

Before Big Data and Hadoop came into picture the term “Analytics” exhibited a stable ground closer to dashboards but now the trend for Analytics seems to be following Big Data and Hadoop.

Let us take a deeper look into each week since 2004

Image

 

Look at the downward spikes occuring around Christmas time. Nobody wants to hear about Big Data or Dashboards during holidays.

And finally, here is a quarterly cyclical view

Image

Click here to view the full interactive Visualizations

Comments Off

Auto Sales Data Visualization by Manufacturer

Data: Edmunds

Image

 

Top Manufacturer

Image

 

Quarterly breakup of units sold by manufacturer

Image

 

View the interactive visualizations

Comments Off

Holiday Sales by category

Image

Comments Off

Fresh and Frozen Fruit consumption – U.S. Bureau of Labor Statistics

The south and the West consume highest amount of Fruits.

Image

Here is more individual breakdown by Quarterly expenditure on Fruits (figures in 100 million)

Image

Image

Image

Comments Off

Visualization on How the undergraduate tuition has increased over the years

Average undergraduate tuition and fees and room and board rates

Source: http://nces.ed.gov/

Image

These figures are inflation adjusted and look how just the tuition fees have increased compared to the Dorm and Board rates

Now comparing the rate increase for 2-year program

Image

So for the 2 year program, the board rates have remained at the same level compared to the dorm rates.

Now check out the interesting graph for 4 year program below

Image

 

Comparing the slope of 2 year Board rates to the 4 year Board rates, the 4 year has significant increase

Image

If price of meals is same for both programs then both 4 year and 2 year programs should have the same slope. So why is the 4 year slope different than 2 year?

Now, let see about the Dorm rates

Image

 

And finally the 4 year vs 2 year Tuition rates

Image

Here is the data table for the above visualization

Comments Off

50 years of killing Deer – Data visualization and analysis

Virginia maintains the summary of Deer kills way back from 1947

The stack bar gives a total view of the killings and how it has grown over the years

Image

 

By comparing the killings on a line chart we see that the female Deer killings has an uptick from 2008 onwards

Image

Comments Off

USA War Casualties

iCasualties.org maintains documented list of all fatalities for Iraq and Afghanistan wars.

Analysing the dataset for Afghanistan, we summarize the results by the year

NOTE: This contains only Afghanistan metrics. We will later update the visuals to reflect Iraq war.

 

Image

USA war fatalities by year

We are approaching the levels of 2002 and hope for the best that we don’t have to suffer another wars.

Here is another view by year and month

 

 

The dataset contains the age of each person died in the war so summarizing by Age

Image

War Deaths by Age

Checking it against the year

Image

Why so many young deaths between age 20 and 30 for the year 2014?

Image

Where did most of the deaths occur?

Image

 

Where were the soldiers from?

Image

Deaths by Rank

 

 

Cause of Death

Attack Types

Image

Image

Image

Helicopter Crash is the one of the top death cause in Non Hostile situations

Comments Off

github DMCA takedown notices in the rise!

Categories: Other

GitHub maintains a list of all DMCA takedown notices along with counteractions and retractions if any.

Analysing all the notices from 2011, it seems that the takedown notices are on the rise.

Year View : Notice the sharp increase in 2014

Image

 

Quarterly view : Now looking at the quarterly breakup, seems like the takedowns are cooling off in the later quarters.

Image

 

So who is issuing these DMCA takedowns?

Here is the complete list of all companies who issued DMCA takedowns

Image

NOTE: The names were extracted from the description text

And here are the counteractions and retractions

Image

See the full list of companies with notice type

So the important question is “Why the DMCA takedown notices have increased?”

One important thing to note is sites like Stackoverflow encourage to replicate the content of the web page from where the original idea/algorithm or source code is copied from. To be honest it is a good thing because lot of times these referring sites become zombies and you don’t want to lose this knowledge. But could it be the case that such non-referenceable source codes end up in GitHub and hence causing the increase in the takedown notices as companies start discovering them?

Comments Off

Fastest growing and rapidly declining job industry

Data source : http://www.bls.gov/emp/tables.htm#occtables

 

Fastest growing job industry

Image

Original Visualization

Most rapidly declining job industry

Image

Rapidly declining jobs link

Comments Off