By the way, somebody suggested Julia, Rust or other languages for data intensive stuff. No, only C or C++ apply here. The reason is that only these 2 languages provide full control over the memory. You can align the data exactly as you want, allocate or free memory at will, etc. Using those tools and knowing the computer architecture and how computers work, you can increase the data density, fully utilize the cache lines and increase substantially the speed. Other languages can’t do that, cause they are design to provide you a way to manage the memory layout.
A lot of people use python, R or mathematica for data mining. The only reason why they do that is because they have libraries which implement most of the stuff, from clustering to neural networks. You just create a small script, and you can play around with models. This is very good for uncovering data patterns and models. Now, if you want to apply those models on large data sets and near real time, you will have to move them to C or C++.
