Author : MD TAREQ HASSAN | Updated : 2022/08/09
What is mapping data flow?
Data flow:
- Data flow allows us to build complex ETL processes that transform data visually
- Data engineers can develop data transformation logic without writing code
- Data flows are executed as activities within Azure Data Factory pipelines that use scaled-out Apache Spark clusters (ADF-managed spark cluster)
Mapping data flow:
- Mapping data flows are visually designed data transformations in Azure Data Factory
- Mapping data flows provide an entirely visual experience with no coding required
- A mapping data flow allows us to build and maintain data transformation graphs that execute on Spark without needing to understand Spark clusters or Spark programming
Notes:
- Azure Data Factory handles all the code translation, path optimization, and execution of your data flow jobs
- If you prefer to code transformations by hand, ADF supports external activities for executing your transformations on compute services such as HDInsight Hadoop, Spark, Data Lake Analytics, and Machine Learning.