This job was posted by https://www.azjobconnection.gov : For more
information, please see: https://www.azjobconnection.gov/jobs/6443031
Development of a big data platform in Hadoop using Spark, Oozie, and
related; Implement and support big data tools, frameworks such as HDFS,
HIVE and streaming technologies such as Kafka and Spark; Set standards
for warehouse and schema design; Write HIVE Queries to process and
manage data in external tables; Automating task of loading the data into
HDFS by developing workflow jobs in Oozie; Develop/build Maven scripts
to integrate them with Jenkins and automate the compilation, deployment
and testing of web applications through XL Release; feed machine
learning models in big data environment; Developing, optimising
hadoop/Spark jobs and automation of hadoop/spark jobs using shell scrip,
event engine tool; Developing scala, python and java based rest API for
batch processing, Real time processing of streaming requests through API
in big data platform; Responsible for unit testing, debugging, verifying
splunk logs and deploying the production ready code and various micro
services for US and International market projects. Migrating big-data
projects to new architectural platform and configuration driven
applications from old platform using technologies Java, Hadoop
Distribution, Map-Reduce, HIVE, Shell Scripting, SQL, REST Services;
Developing and maintaining the big-data pipelines (batch and real-time)
from end-to-end flow. Required Masters inInfo.Sys/CS/Inf.Tech/related
plus 6 months experience asProgrammer/System
Analyst/Soft.Dev/Soft.Engg/related. Travel may be required.