This job board retrieves part of its jobs from: Toronto Jobs | Emplois Montréal | IT Jobs Canada

The New York State wants YOU to start a career here!

To post a job, login or create an account |  Post a Job

Hadoop Lead


This is a Full-time position in Buffalo, NY posted February 20, 2021.

Hadoop Lead Buffalo NY (Remote until Covid) Participate in detailed technical design, development, implementation and support of Big Data applications. Maintain standards compliance and ensure development artifacts are in alignment with patternsframeworks designed by software engineering teams. Skillsets Familiarity with Spark programming paradigms (batch and stream data processing). Understanding of different data abstraction objects used in spark for different use cases, use of optimal data format and other optimization techniques. Strong programing skills in at least one of the following languages Java, Scala, Pyspark. Familiarity with a scripting language like Python as well as UnixLinux shells. Strong knowledge of writing optimized Spark Hive sql and experience to tune poor performing queries. Outstanding programming and debugging skills. Strong knowledge of common algorithms and data structures. Good understanding of job scheduling and workflow orchestration through enterprise scheduling tools preferably CAAutomic. Strong experience with SQL and relational databases like PostgreSQL, MySQL, Teradata, SQL Server and Oracle. Strong handson experience working on Hadoop ecosystem such as Hive, Impala, Pig, Sqoop, Hue, HBase, etc. Familiarity with one or more stream processing queuing technologies like Spark Streaming, Kafka, Kinesis, Flink, etc. preferred. Familiarity and prior experience with AgileScrum development methodologies. Prior Experience deploying to cloud platforms, preferably Azure or AWS Cloud Prior experience in Continuous IntegrationContinuous Delivery tools and pipelines such as Jenkins, Github, Maven, Gradle, etc.