DataMine Lab is a data mining and business intelligence company We provide data and software consulting to the advertising industry
- e-mail: [email protected]
- September 16, 2012YCSB run against HBase 0.92 on Amazon Elastic MapReduce
- May 4, 2012BigData events
- November 21, 2011R Analytics in the Cloud
- October 3, 2011BigData London events (and a ticket giveaway)
- September 6, 2011Calculating unique visitors in Hadoop and Hive
YCSB run against HBase 0.92 on Amazon Elastic MapReduce
September 16, 2012 by Krystian Nowak, no comments
In this post we will show you how in simple steps using Yahoo! Cloud Serving Benchmark: https://github.com/dataminelab/YCSB you can run benchmarks against HBase 0.92 cluster deployed automatically by Amazon Elastic MapReduce and what measurements and comparisons you can obtain while choosing among different available instance types.
May 4, 2012 by Radek Maciaszek, 2 comments
We observe an explosion of BigData events. While half a year ago London hosted maybe one interesting meetup a month nowadays there is rarely a week without few of them. Supply is keeping up with demand.
There is an increasing number of monthly meetups: BigData London, HUG UK, Data Science London, London R, Cassandra London, Neo4j London, London MongoDB User Group, Oracle BigData, Data Visualisation London, Big Data Debate, DeNormalised London, LonData, CloudComputing.
Upcoming conferences that are worth mentioning:
- Skillsmatter Progressive NoSQL Tutorial (9-11th May 2012)
- Berlin Buzzworlds – only couple hours from London via Eurostar (4-5th June 2012)
- Big Data Analytics 2012 (20th June 2012)
- Big Data Summit 2012 (28th June 2012)
- Big Data World Europe (19-20th September 2012)
- DeNormalised NoSQL Conference London (20-21st September 2012)
- Big Data Congress (7th November 2012)
We just had a London BigData week that was full of meetings and hackatons dedicated to Hadoop, Visualisations and NoSQL. In case you missed the last Big Data week you are for a treat – simply like us on Facebook to have a chance of winning one ticket (worth £495) for 3 days of SkillsMatter NoSQL tutorials.
There are as well few online places where every data scientist can improve or challenge their skills:
If you know of anything interesting coming up in London, let us know in the comments.
R Analytics in the Cloud
November 21, 2011 by Radek Maciaszek, no comments
Last week I was invited to Big Data London to talk about “R Analytics in the Cloud”. As a case study, I presented the ageing project I’ve been working on as part of my Masters studies at Birkbeck, University of London. Ageing is one of the fundamental mysteries in biology and many scientists are already studying this process. I am excited to be part of the research group led by Eugene Schuster at UCL Institute of Healthy Ageing. This project has also given me the chance to use some of my Hadoop experience in the academic field.
Bioinformatics is the science of applying information technology to biology in order to understand the latter. There are numerous ways in which computers can aid biologists. In this particular project, we have been using microarrays to find the connection between different genes. The use of microarray technologies has enabled us to detect changes to gene expression across the genome in thousands of experiments with hundreds of species. However, interpreting the changes identified in these experiments has been hampered by a lack of knowledge of the gene function. Even in highly studied genomes, approximately 50-60% of genes will be assigned functions, yet less than 30% will be annotated with a highly specific function. Little of the annotation will have been observed in experiments conducted with the species of interest, as most gene function annotation is based on annotations assigned to orthologous genes taken from experiments done with other species, such as yeast and mammalian cell culture.
We are interested in building a better understanding of gene function in the worm C. elegans by harnessing the large quantity of experimental microarray data in the public database. Currently, we have a database of over fifty curated experiments. With this, we attempt to assign putative functions to genes based on the expression profile across experiments in the public repositories. My role in this project is to help expand the number of curated experiments in the database and study the functions of approximately 1000 genes known to be regulated in long-lived worms, to try to understand the functions of these genes, e.g. by showing experimental evidence of a role in nutrient sensing, innate immunity or stress response.
Here are the slides from the presentation. Refer to slides 10 and 11 to see how to migrate your R application to the cloud in just 3 lines of code:
Oh, and did I mention how cool our lab is? Have a look at the following ad, which was made at UCL just a couple of metres from my desk.
Full disclosure: DataMine Lab is in no way affiliated with Birkbeck or UCL and the above project is part of my individual bioinformatics studies.