Data Engineer (Azure, ADF, Big Data, ETL) (QR8914)

Data Engineer (Azure, ADF, Big Data, ETL) (QR8914)



  • Omschrijving

    Functie: Data Engineer (Azure, ADF, Big Data, ETL) (QR8914)
    Start: 4-7-2022, 32-40 uur per week
    Periode: 3 mnd+
    Omgeving: Omgeving Amsterdam

    For our client in Amsterdam we are looking for a Data Engineer (Azure, ADF, Big Data, ETL)

    The person should have experience in technologies for distributed data computing: ADF, Azure Data Lake, Azure Data Lake Analytics, Azure database, Azure data warehouse, Delta Lake and Azure DataBricks.

    Should have:
    • Should have in-Depth knowledge and experience in design, development of Big Data projects that perform large scale Distributed Data Processing/Analytics Microsoft Azure.
    • Experience in roles of Big Data Developer, ETL Lead – specializing in Apache Spark, Kafka and Hbase.
    • Developed applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
    • Experience in Data Platform: Azure Databricks, Azure Data Lake
    • Experience in Data Pipelines/Workflow Management Tools: Apache Airflow, Azure Data Factory
    • Knowing how to code using the object-oriented paradigm(See below for programming languages)
    • Experience in programming/coding with Python, PySpark, SQL, Scala, Shell Scripting and building Python packages/wheels
    • Experience with using file formats and technologies like Apache Arrow, Parquet and Delta Lake
    • Experience in automation of deployments (Azure and On-Premises) using using CI/CD tools like Jenkins, Github, Azure DevOps, Github Actions
    • Knowledge on Azure cloud principles like IAM, Active Directory, PIM etc.
    • Experience in Data visualization tools like Power BI, Tableau
    • Experience in working with Relational Databases(RDBMS) and No SQL Databases
    • Experience in logging & monitoring is preferred, not required
    • Worked with Test, QA and DevOps teams to automate the test cases and integrate them with scheduled auto deployment pipeline to increase test coverage.
    • For Senior Roles, need to understand Cost management in Azure.
    • Working experiences in Agile Scrum team.

  • Omschrijving Freelancer

  • Gebruikersnaam

    U dient in te loggen om gebruik te maken van

  • Opdrachtrubriek

    ICT (applicaties en software)

  • Locatie Opdracht


  • Provincie


  • Bijlagen

  • Gewenste Reacties

    Bedrag per uur

  • Verspreiden via social media