Statisticians, Scripts, and Chaos: My Journey Back to the 90s
\n \t
\n
<\/BR>
\n
\n \t
\n
<\/BR><\/p>\n
We often hear a lot about how data science teams can lack statistical expertise and how this can lead to flawed analyses or misinterpretation of results. It\u2019s a valid concern, and the dangers are real. But let me tell you, there\u2019s another side of the coin that had me saying, \u201cHoly bleep.\u201d<\/p>\n
This year, I joined a project where the team is dominated by statisticians and economists. Sounds like a data science dream team, right? Not so fast. It feels like I hopped into a time machine and landed in the 90s. Git? Never heard of it. Instead, we\u2019ve got the old-school hierarchy of script_v1, script_final_version_1, script_final_version_2, all the way to script_final_version_n. It’s a wild ride.<\/p>\n
Code reviews? Absolutely nonexistent. Every script is its own handcrafted masterpiece, riddled with what I can only describe as “surprise features” in the preprocessing pipeline. Bugs aren\u2019t bugs, apparently. \u201cIf you just pay close attention and read your code twice, you\u2019ll see there\u2019s no issue,\u201d they tell me. Uh, sure. I don\u2019t trust a single output right now because I know that behind every analysis bugs are having the party of their lives. <\/p>\n
Chances are, statisticians have absolutely no idea how a modern database actually works, have never heard of a non-basic data structure like a HyperLogLog, and have likely never wrestled with a truly messy real-world dataset.<\/p>\n<\/p><\/div>\n
submitted by \/u\/Raz4r <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span>\n<\/div>\n