Big Data Hadoop EngineerLocation: Pleasanton, CA
Posted On: 02/22/2021
Requirement Code: 43810
A. Resource Requirements, Skills, Knowledge and Abilities:
Supplier shall ensure that all resources assigned to the project have the minimum skills requirement to render the services in a competent and efficient manner.
Technical Knowledge and Skills:
4+ years of hands-on Development, Deployment and production Support experience in Big Data environment.
4-5 years of programming experience in Java, Scala, Python.
Proficient in SQL and relational database design and methods for data retrieval.
Knowledge of NoSQL systems like HBase or Cassandra
Hands-on experience in Cloudera Distribution 6.x
Hands-on experience in creating, indexing Solr collections in Solr Cloud environment.
Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL.
Must have experience with developing Hive QL, UDF's for analyzing semi structured/structured datasets.
Must have experience with Spring framework, Web Services, REST API's and MicroServices.
Hands-on experience ingesting and processing various file formats like Avro/Parquet/Sequence Files/Text Files etc.
Must have working experience in the data warehousing and Business Intelligence systems.
Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs.
Successful track record of building automation scripts/code using Java, Bash, Python etc. and experience in production support issue resolution process.
Experience in building ML models using MLLib or any ML tools.
Hands-on experience working in Real-Time analytics like Spark/Kafka/Storm
Experience with Graph Databases like Neo4J, Tiger Graph, Orient DB
Agile development methodologies.
The Consultant resource(s) shall possess most of the following skills:
Strong analytical skills with the ability to analyze information and identify and formulate solutions to problems.
Provide more in-depth analysis with a high-level view of goals and end deliverables.
Complete work within a reasonable time frame under the supervision of a manager or team lead.
Plan and manage all aspects of the support function.
Extensive knowledge of and proven experience with data processing systems, and methods of developing, testing and moving solutions to implementation.
Strong knowledge in project management practices and ability to document processes and procedures as needed.
Work collaboratively with other support team members and independently on assigned tasks and deliverables with minimum supervision
Communicate effectively with users at all levels, from data entry technicians up to senior management, verbally and in writing.
Self-motivated, working closely and actively communicating with team members to accomplish time critical tasks and deliverables
Ask questions and share information gained with other support team members, recording and documenting this knowledge
Elicit and gather user requirements and/or problem description information, and record this information accurately
Listen carefully and act upon user requirements
Convey and explain complex problems and solutions in an understandable language to both technical and non-technical persons
Present technical solutions to management and decision makers
Follow the lead of others on assigned projects as well as take the lead when deemed appropriate
Take responsibility for the integrity of the solution
Act with integrity