In 2014, I shared the specifications of a 6-core 64GB RAM desktop system that I purchased at around USD 2,000. Since then, I added NVidia Titan X to it for deep learning at additional USD 1,000, and it served me well.
However, as other team members started joining me on data science competitions and deep learning competitions got more popular, my team decided to build a more powerful desktop system.
The specifications of the new system that we built are as follows:
Total cost including tax and shipping was around USD 7,000. Depending on the budget, you can go down to 2 (-USD 1,520) 1080 Ti GPU cards instead of 4, or 64GB (-USD 399) instead of 128GB RAM, and still have a decent system at around USD 5,000.
You can find the full part lists here.