Vinay BadhanUnit testing for pyspark jobsWhy unit testing is required for pyspark jobs?Apr 9, 2023Apr 9, 2023
Vinay BadhanUsing Apache Hudi snapshot queries with PySparkProblem statement:Jun 19, 2021Jun 19, 2021
Vinay BadhanOptimisations and checkpointing for spark streaming jobI currently work at HackerRank, we collect data points from various sources of our platform, enterprise and community(developers). We…Mar 15, 2021Mar 15, 2021
Vinay BadhanSpark Streaming MonitoringSpark Streaming is an extension of the core Spark API that enables scalable, high-throughput, fault-tolerant stream processing of live…Oct 8, 2020Oct 8, 2020
Vinay BadhanPublish/Subscribe (Pub/Sub) Pattern in BashWhen a Bash project turns into a library, it can become difficult to add new functionality. Function names, variables and parameters…Apr 14, 2019Apr 14, 2019
Vinay BadhanPitfalls in BashWhitespaces when assigning variables Whitespace matters when assigning variables.Apr 14, 2019Apr 14, 2019
Vinay BadhanFile execution sequence.bash_profile, .bash_login, .bashrc and .profile all do pretty much the same thing: set up and define functions, variables and the sorts.Apr 13, 2019Apr 13, 2019
Vinay BadhanChange Shell in Mac/LinuxFind the current shell: There are a few ways to determine the current shellApr 13, 2019Apr 13, 2019
