Message: Please make sure that the type of parameter matches with type of value passed in.Recommendation: Remove the partitioned root directory from the source path and read it through separate source transformation.Cause: The source path has either multiple partitioned directories or a partitioned directory that has another file or non-partitioned directory.a=10/b=20, //c=10/d=30) or partitioned directory with other file or non-partitioned directory (for example //a=10/b=20, /Directory 2/file1), remove partition root directory from source path and read it through separate source transformation. Message: The specified source path has either multiple partitioned directories (for e.g.Recommendation: Contact the Microsoft product team for more details about this problem.Įrror code: DF-Executor-PartitionDirectoryError.Cause: The maximum number of uncommitted blocks in a blob is 100,000.Message: The uncommitted block count cannot exceed the maximum limit of 100,000 blocks.Consider upgrading the column type to the latest type.Įrror code: DF-Executor-BlockCountExceedsLimitError Recommendation: INT96 is a legacy timestamp type that's not supported by Azure Data Factory data flow.Please consider upgrading the column type to the latest types. Message: INT96 is legacy timestamp type which is not supported by ADF Dataflow.Recommendation: Set an alias if you're using a SQL function like min() or max().Message: Column name needs to be specified in the query, set an alias if using a SQL function.Recommendation: Use the correct data type.Cause: Data isn't in the expected format.Message: Converting to a date or time failed due to an invalid character.In the absence of a broadcast join, use a larger cluster if this error occurs.
Large Azure SQL Data Warehouse tables and source files aren't typically good choices. Recommendation: Turn off the broadcast option or avoid broadcasting large data streams for which the processing can take more than 60 seconds. If a broadcast join isn't used, the default broadcast by dataflow can reach the same limit. On the broadcast join, the stream chosen for broadcast is too large to produce data within this limit. If you intend to broadcast join option to improve performance then make sure broadcast stream can produce data within 60 secs in debug runs and 300 secs in job runs.Ĭause: Broadcast has a default timeout of 60 seconds in debug runs and 300 seconds in job runs. Message: Broadcast join timeout error, you can choose 'Off' of broadcast option in join/exists/lookup transformation to avoid this issue. To do so, you can use the Debug > Use Activity Runtime option to use the Azure IR defined in your Execute Data Flow pipeline activity. You can extend the timeout to the 300-second timeout of a triggered run. That's because Azure Data Factory throttles the broadcast timeout to 60 seconds to maintain a faster debugging experience. If you're running the data flow in a debug test execution from a debug pipeline run, you might run into this condition more frequently. For the best performance in data flows, we recommend that you allow Spark to broadcast by using Auto and use a memory-optimized Azure IR. If Auto is set, or if you're manually setting the left or right side to broadcast under Fixed, you can either set a larger Azure integration runtime (IR) configuration or turn off broadcast. The default option for broadcast is Auto. Recommendation: Check the Optimize tab on your data flow transformations for join, exists, and lookup. The stream chosen for broadcast is too large to produce data within this limit. Message: Broadcast join timeout error, make sure broadcast stream produces data within 60 secs in debug runs and 300 secs in job runsĬause: Broadcast has a default timeout of 60 seconds on debug runs and 300 seconds on job runs. On the source transformation that's using a JSON dataset, expand JSON Settings and turn on Single Document.