FULL-STACK DATA ENGINEER (PYTHON, AWS, AIRFLOW)
Descrição da oferta de emprego
We emphasize AI as a fundamental tool for business growth and competitive advantage, catering to various industries with a focus on data-driven decision-making and operational efficiency.
We are seeking a versatile Full-Stack Data Engineer with expertise in Python programming, AWS services, DevOps practices, and a solid understanding of both software and data engineering principles.
This role is pivotal in building and maintaining scalable data infrastructure, automating data processes, and contributing to the development of software solutions that manage and analyze large data sets.
Responsibilities.
Design, implement, and manage microservices architecture using AWS cloud services.
Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG.
Implement DevOps strategies for continuous integration and continuous deployment (CI/CD) to improve software development processes.
Write high-quality, maintainable code in Python to support data operations.
Integrate various internal and external data sources (including SharePoint, One Drive, One Note, etc.) to create a unified data platform.
Build tools for document ingestion, dynamic file previewing, and tabular data extraction.
Manage graph database systems and support large text corpus management.
Implement security controls to maintain data integrity and compliance.
Use Large Language Models for keyword extraction and data analysis to gain insights.
\n Proven experience with Python programming and AWS services (EC2, RDS, S3, Lambda, etc.).
Proficient in managing and orchestrating workflows with Airflow.
Solid understanding of DevOps principles, including experience with CI/CD tools and practices.
Demonstrated experience with data engineering and familiarity with graph database management.
Expertise in data processing, corpus management, and keyword extraction.
Strong problem-solving abilities, attention to detail, and organizational skills.
Excellent communication and teamwork skills to collaborate with cross-functional teams.
Minimum overlap of 5 hours with EST time zone during working hours.
Detalhes da oferta
- Xenon7
- Em todo o Brasil
- Indeterminado - Indeterminado
- Indeterminado
- 02/04/2024
- 01/07/2024
Net, python e php; - desenvolvimento de scripts e automatização de processos em powershell e bash script; - instalação e adminstração de bancos de dados: mysql ou microsoft sql, azure data explorer ou radoop; - instalação e adminsitração de infraestrutra de frontend e backend; - utilização de ferramentas......
Metodologia ágil: formato de squads, ferramenta azure e jiraescopo:- nas implantações das demandas o profissional será responsável pelos artefatos da gmud (documentações da mudanças, scripts, roteiro e evidências de testes, validação dos usuários, procedimentos de rollback etc... ), bem como acompanharpresencialmente......
Homologação/acompanhamento de atualizações com usuários... net, com c#, mvc e angular... sustentação de sistemas de produção... elaboração de documentações de gmud... teste em ambientes de desenvolvimento e homologação... net c# mvc, angular , api´s, webservices, json, xml,• experiência em oracle pl/sql......
Teste: realizar testes para garantir funcionalidade e segurança do software, participando de revisões de código... benefícios salário de 300... horário de trabalho segunda a sexta, das 9:00 às 17:00 responsabilidades planejamento: contribuir para definir requisitos e roteiro do projeto em colaboração......
Maintenance) of raw data from various data sources into the data warehouse;• automation and optimization of data imports;• preparation and normalization of data;• creating sap bo reports with multiple data sources;• at least 3 year’s experience in data analysis and/or reporting, ideally in a data warehouse......
Submit recorded data promptly... reliable internet connection for data submission... ready to make your voice heard worldwide? join us in this unique opportunity to be part of a revolutionary ai initiative! french (canadian)-speaking audio data collector (recruitee... position type: independent contractor......
Submit recorded data promptly... reliable internet connection for data submission... outros dados de posição french (canadian)-speaking audio data collector (recruitee... position type: independent contractorproject type: one-time projectindustry: artificial intelligence (ai) about the opportunity: are......
Lead a portfolio of diverse technology projects and a team of developers with deep experience in distributed microservices, and full stack systems to create solutions that help meet regulatory needs for the company • share your passion for staying on top of tech trends, experimenting with and learning......
The ability to organize and analyze data effectively is also important... the job involves extracting data from pdf documents and accurately transferring it into excel worksheets... relevant skills include: - proficiency in using pdf to excel conversion tools - strong attention to detail - data organization......
Conhecimentos desejáveis: cloud computing (azure, aws e/ou google); fluência no idioma inglês ou espanhol... requisitos do trabalho conhecimentos avançados: gestão e versionamento do código fonte e objetos do projeto; integração contínua do projeto (ci); implantação contínua do projeto (cd); técnicas......
