Common fault codes and messages

  • Article
  • 25 minutes to read

This commodity lists mutual mistake codes and letters reported past mapping data flows in Azure Data Manufactory, along with their associated causes and recommendations.

Error code: DF-Executor-SourceInvalidPayload

  • Bulletin: Data preview, debug, and pipeline data menstruum execution failed considering container does not be
  • Cause: A dataset contains a container that doesn't be in storage.
  • Recommendation: Make certain that the container referenced in your dataset exists and tin be accessed.

Error code: DF-Executor-SystemInvalidJson

  • Message: JSON parsing error, unsupported encoding or multiline
  • Cause: Possible problems with the JSON file: unsupported encoding, corrupt bytes, or using JSON source as a single document on many nested lines.
  • Recommendation: Verify that the JSON file's encoding is supported. On the source transformation that's using a JSON dataset, expand JSON Settings and plow on Unmarried Document.

Mistake code: DF-Executor-BroadcastTimeout

  • Message: Circulate join timeout error, make certain broadcast stream produces information within 60 secs in debug runs and 300 secs in chore runs

  • Cause: Broadcast has a default timeout of 60 seconds on debug runs and 300 seconds on chore runs. The stream chosen for circulate is also big to produce information inside this limit.

  • Recommendation: Cheque the Optimize tab on your data menstruation transformations for bring together, exists, and lookup. The default option for broadcast is Auto. If Auto is set, or if you lot're manually setting the left or right side to broadcast nether Fixed, you can either set a larger Azure integration runtime (IR) configuration or turn off broadcast. For the all-time operation in data flows, nosotros recommend that yous allow Spark to broadcast by using Auto and use a retention-optimized Azure IR.

    If you're running the data flow in a debug test execution from a debug pipeline run, you might encounter this condition more frequently. That's considering Azure Information Manufacturing plant throttles the broadcast timeout to lx seconds to maintain a faster debugging experience. You can extend the timeout to the 300-second timeout of a triggered run. To do then, y'all tin can use the Debug > Use Activity Runtime option to use the Azure IR defined in your Execute Information Menses pipeline activeness.

  • Message: Broadcast join timeout error, you lot can cull 'Off' of broadcast option in join/exists/lookup transformation to avoid this issue. If you intend to broadcast join option to meliorate operation, then make sure broadcast stream tin can produce data within 60 secs in debug runs and 300 secs in job runs.

  • Cause: Broadcast has a default timeout of threescore seconds in debug runs and 300 seconds in job runs. On the broadcast join, the stream chosen for broadcast is as well big to produce data within this limit. If a broadcast join isn't used, the default circulate by dataflow tin achieve the same limit.

  • Recommendation: Turn off the broadcast choice or avert broadcasting large data streams for which the processing can have more than 60 seconds. Cull a smaller stream to broadcast. Large Azure SQL Data Warehouse tables and source files aren't typically good choices. In the absence of a broadcast bring together, utilise a larger cluster if this error occurs.

Error code: DF-Executor-Conversion

  • Bulletin: Converting to a date or time failed due to an invalid character
  • Cause: Data isn't in the expected format.
  • Recommendation: Employ the correct data type.

Error code: DF-Executor-InvalidColumn

  • Message: Column name needs to be specified in the query, ready an allonym if using a SQL function
  • Cause: No cavalcade name is specified.
  • Recommendation: Ready an allonym if you're using a SQL function like min() or max().

Mistake code: DF-Executor-DriverError

  • Bulletin: INT96 is legacy timestamp type, which is not supported by ADF Dataflow. Please consider upgrading the column type to the latest types.
  • Cause: Driver mistake.
  • Recommendation: INT96 is a legacy timestamp type that's non supported by Azure Data Factory data flow. Consider upgrading the cavalcade blazon to the latest type.

Error code: DF-Executor-BlockCountExceedsLimitError

  • Message: The uncommitted block count cannot exceed the maximum limit of 100,000 blocks. Check hulk configuration.
  • Cause: The maximum number of uncommitted blocks in a hulk is 100,000.
  • Recommendation: Contact the Microsoft product squad for more than details most this problem.

Error lawmaking: DF-Executor-PartitionDirectoryError

  • Message: The specified source path has either multiple partitioned directories (for example, <Source Path>/<Partition Root Directory one>/a=10/b=20, <Source Path>/<Partitioning Root Directory 2>/c=x/d=xxx) or partitioned directory with other file or non-partitioned directory (for example <Source Path>/<Partition Root Directory 1>/a=10/b=20, <Source Path>/Directory 2/file1), remove segmentation root directory from source path and read it through separate source transformation.
  • Cause: The source path has either multiple partitioned directories or a partitioned directory that has another file or not-partitioned directory.
  • Recommendation: Remove the partitioned root directory from the source path and read it through separate source transformation.

Error lawmaking: DF-Executor-InvalidType

  • Message: Please make sure that the type of parameter matches with type of value passed in. Passing float parameters from pipelines isn't currently supported.
  • Cause: Data types are incompatible between the declared blazon and the actual parameter value.
  • Recommendation: Bank check that the parameter values passed into the information menses lucifer the declared type.

Error code: DF-Executor-ParseError

  • Message: Expression cannot be parsed.
  • Crusade: An expression generated parsing errors because of incorrect formatting.
  • Recommendation: Check the formatting in the expression.

Error code: DF-Executor-SystemImplicitCartesian

  • Bulletin: Implicit cartesian product for INNER join is not supported, utilise Cross JOIN instead. Columns used in bring together should create a unique fundamental for rows.
  • Cause: Implicit cartesian products for INNER joins between logical plans aren't supported. If yous're using columns in the bring together, create a unique fundamental.
  • Recommendation: For non-equality based joins, use Cross Join.

Mistake code: GetCommand OutputAsync failed

  • Message: During Data Flow debug and information preview: GetCommand OutputAsync failed with ...
  • Cause: This error is a back-end service mistake.
  • Recommendation: Retry the functioning and restart your debugging session. If retrying and restarting doesn't resolve the problem, contact customer back up.

Fault code: DF-Executor-OutOfMemoryError

  • Message: Cluster ran into out of memory issue during execution, please retry using an integration runtime with bigger core count and/or memory optimized compute type
  • Cause: The cluster is running out of memory.
  • Recommendation: Debug clusters are meant for development. Employ data sampling and an appropriate compute type and size to run the payload. For functioning tips, see Mapping information flow performance guide.

Mistake code: DF-Executor-illegalArgument

  • Message: Please make sure that the access key in your Linked Service is right
  • Crusade: The account name or access key is incorrect.
  • Recommendation: Ensure that the account name or access key specified in your linked service is correct.

Error lawmaking: DF-Executor-ColumnUnavailable

  • Message: Cavalcade name used in expression is unavailable or invalid.
  • Cause: An invalid or unavailable cavalcade proper noun is used in an expression.
  • Recommendation: Check the column names used in expressions.

Error code: DF-Executor-OutOfDiskSpaceError

  • Message: Internal server error
  • Crusade: The cluster is running out of deejay space.
  • Recommendation: Retry the pipeline. If doing and so doesn't resolve the problem, contact customer back up.

Error lawmaking: DF-Executor-StoreIsNotDefined

  • Message: The store configuration is not defined. This error is potentially caused by invalid parameter assignment in the pipeline.
  • Cause: Invalid shop configuration is provided.
  • Recommendation: Check the parameter value consignment in the pipeline. A parameter expression may contain invalid characters.

Error lawmaking: 4502

  • Bulletin: There are substantial concurrent MappingDataflow executions that are causing failures due to throttling under Integration Runtime.
  • Cause: A large number of Data Flow activity runs are occurring meantime on the integration runtime. For more information, see Azure Data Mill limits.
  • Recommendation: If you lot want to run more Data Menstruation activities in parallel, distribute them across multiple integration runtimes.

Error code: 4510

  • Message: Unexpected failure during execution.
  • Crusade: Since debug clusters work differently from task clusters, excessive debug runs could wear the cluster over time, which could cause memory issues and abrupt restarts.
  • Recommendation: Restart Debug cluster. If you are running multiple dataflows during debug session, use activity runs instead because activity level run creates split up session without taxing main debug cluster.

Error code: InvalidTemplate

  • Message: The pipeline expression cannot exist evaluated.
  • Crusade: The pipeline expression passed in the Data Flow activeness isn't existence processed correctly considering of a syntax error.
  • Recommendation: Check information menses activity name. Check expressions in activity monitoring to verify the expressions. For example, data flow activity name can not have a infinite or a hyphen.

Fault code: 2011

  • Message: The activity was running on Azure Integration Runtime and failed to decrypt the credential of information store or compute continued via a Self-hosted Integration Runtime. Delight bank check the configuration of linked services associated with this activeness, and make sure to use the proper integration runtime type.
  • Cause: Data flow doesn't support linked services on self-hosted integration runtimes.
  • Recommendation: Configure data menstruation to run on a Managed Virtual Network integration runtime.

Error code: DF-Xml-InvalidValidationMode

  • Message: Invalid xml validation mode is provided.
  • Cause: An invalid XML validation mode is provided.
  • Recommendation: Cheque the parameter value and specify the correct validation mode.

Fault lawmaking: DF-Xml-InvalidDataField

  • Bulletin: The field for decadent records must be string blazon and nullable.
  • Cause: An invalid data blazon of the column \"_corrupt_record\" is provided in the XML source.
  • Recommendation: Make sure that the column \"_corrupt_record\" in the XML source has a string information type and nullable.

Mistake code: DF-Xml-MalformedFile

  • Message: Malformed xml with path in FAILFAST manner.
  • Cause: Malformed XML with path exists in the FAILFAST mode.
  • Recommendation: Update the content of the XML file to the right format.

Error code: DF-Xml-InvalidReferenceResource

  • Message: Reference resources in xml data file cannot be resolved.
  • Crusade: The reference resources in the XML information file cannot be resolved.
  • Recommendation: Yous should cheque the reference resource in the XML information file.

Error code: DF-Xml-InvalidSchema

  • Message: Schema validation failed.
  • Cause: The invalid schema is provided on the XML source.
  • Recommendation: Check the schema settings on the XML source to make sure that it is the subset schema of the source data.

Mistake code: DF-Xml-UnsupportedExternalReferenceResource

  • Message: External reference resource in xml data file is not supported.
  • Cause: The external reference resources in the XML data file is not supported.
  • Recommendation: Update the XML file content when the external reference resource is not supported at present.

Error code: DF-GEN2-InvalidAccountConfiguration

  • Bulletin: Either 1 of account key or tenant/spnId/spnCredential/spnCredentialType or miServiceUri/miServiceToken should be specified.
  • Cause: An invalid credential is provided in the ADLS Gen2 linked service.
  • Recommendation: Update the ADLS Gen2 linked service to accept the right credential configuration.

Mistake code: DF-GEN2-InvalidAuthConfiguration

  • Message: Only one of the iii auth methods (Central, ServicePrincipal and MI) tin be specified.
  • Cause: Invalid auth method is provided in ADLS gen2 linked service.
  • Recommendation: Update the ADLS Gen2 linked service to have one of three hallmark methods that are Fundamental, ServicePrincipal and MI.

Error code: DF-GEN2-InvalidServicePrincipalCredentialType

  • Message: Service master credential type is invalid.
  • Cause: The service principal credential type is invalid.
  • Recommendation: Please update the ADLS Gen2 linked service to set the right service principal credential type.

Error lawmaking: DF-Blob-InvalidAccountConfiguration

  • Message: Either ane of business relationship key or sas token should be specified.
  • Crusade: An invalid credential is provided in the Azure Blob linked service.
  • Recommendation: Apply either account primal or SAS token for the Azure Blob linked service.

Error code: DF-Blob-InvalidAuthConfiguration

  • Message: Only ane of the ii auth methods (Key, SAS) can be specified.
  • Crusade: An invalid hallmark method is provided in the linked service.
  • Recommendation: Utilize key or SAS authentication for the Azure Blob linked service.

Mistake code: DF-Cosmos-PartitionKeyMissed

  • Message: Partition central path should be specified for update and delete operations.
  • Cause: The partition key path is missed in the Azure Cosmos DB sink.
  • Recommendation: Use the providing partitioning cardinal in the Azure Creation DB sink settings.

Mistake lawmaking: DF-Cosmos-InvalidPartitionKey

  • Message: Segmentation key path cannot be empty for update and delete operations.

  • Crusade: The partition cardinal path is empty for update and delete operations.

  • Recommendation: Use the providing partition key in the Azure Cosmos DB sink settings.

  • Message: Partition key is not mapped in sink for delete and update operations.

  • Cause: An invalid partition key is provided.

  • Recommendation: In Cosmos DB sink settings, utilise the right partition key that is aforementioned as your container's partition key.

Error code: DF-Cosmos-IdPropertyMissed

  • Message: 'id' property should be mapped for delete and update operations.
  • Cause: The id belongings is missed for update and delete operations.
  • Recommendation: Make sure that the input information has an id column in Cosmos DB sink settings. If no, use select or derive transformation to generate this column before sink.

Error code: DF-Cosmos-InvalidPartitionKeyContent

  • Message: partition central should start with /.
  • Cause: An invalid partition cardinal is provided.
  • Recommendation: Ensure that the partition cardinal kickoff with / in Cosmos DB sink settings, for example: /movieId.

Error code: DF-Cosmos-InvalidConnectionMode

  • Message: Invalid connectedness mode.
  • Cause: An invalid connexion mode is provided.
  • Recommendation: Confirm that the supported mode is Gateway and DirectHttps in Cosmos DB settings.

Error lawmaking: DF-Creation-InvalidAccountConfiguration

  • Bulletin: Either accountName or accountEndpoint should exist specified.
  • Cause: Invalid business relationship information is provided.
  • Recommendation: In the Cosmos DB linked service, specify the business relationship proper noun or account endpoint.

Fault code: DF-Github-WriteNotSupported

  • Message: GitHub store does non permit writes.
  • Cause: The GitHub shop is read merely.
  • Recommendation: The shop entity definition is in some other place.

Error lawmaking: DF-PGSQL-InvalidCredential

  • Message: User/password should be specified.
  • Cause: The User/password is missed.
  • Recommendation: Make sure that you take right credential settings in the related PostgreSQL linked service.

Mistake code: DF-Snowflake-InvalidStageConfiguration (only blob storage can be used as stage)

  • Message: Simply blob storage type can be used as stage in snowflake read/write operation.

  • Crusade: An invalid staging configuration is provided in the Snowflake.

  • Recommendation: Update Snowflake staging settings to ensure that just Azure Blob linked service is used.

  • Message: Snowflake stage properties should exist specified with Azure Blob + SAS authentication.

  • Crusade: An invalid staging configuration is provided in the Snowflake.

  • Recommendation: Ensure that only the Azure Hulk + SAS authentication is specified in the Snowflake staging settings.

Fault code: DF-Snowflake-InvalidDataType

  • Message: The spark blazon is not supported in snowflake.
  • Cause: An invalid information blazon is provided in the Snowflake.
  • Recommendation: Please use the derive transformation before applying the Snowflake sink to update the related cavalcade of the input data into the string type.

Error code: DF-Hive-InvalidBlobStagingConfiguration

  • Message: Blob storage staging properties should be specified.
  • Cause: An invalid staging configuration is provided in the Hive.
  • Recommendation: Please check if the account key, account proper noun and container are ready properly in the related Hulk linked service, which is used as staging.

Fault lawmaking: DF-Hive-InvalidGen2StagingConfiguration

  • Bulletin: ADLS Gen2 storage staging simply back up service principal primal credential.

  • Cause: An invalid staging configuration is provided in the Hive.

  • Recommendation: Please update the related ADLS Gen2 linked service that is used equally staging. Currently, only the service principal key credential is supported.

  • Message: ADLS Gen2 storage staging backdrop should be specified. Either one of key or tenant/spnId/spnKey or miServiceUri/miServiceToken is required.

  • Cause: An invalid staging configuration is provided in the Hive.

  • Recommendation: Update the related ADLS Gen2 linked service with right credentials that are used equally staging in the Hive.

Error code: DF-Hive-InvalidDataType

  • Message: Unsupported Column(south).
  • Cause: Unsupported Column(s) are provided.
  • Recommendation: Update the column of input data to match the information type supported by the Hive.

Error code: DF-Hive-InvalidStorageType

  • Message: Storage type can either be hulk or gen2.
  • Cause: But Azure Blob or ADLS Gen2 storage blazon is supported.
  • Recommendation: Choose the right storage type from Azure Blob or ADLS Gen2.

Fault code: DF-Delimited-InvalidConfiguration

  • Bulletin: Either ane of empty lines or custom header should be specified.
  • Cause: An invalid delimited configuration is provided.
  • Recommendation: Please update the CSV settings to specify one of empty lines or the custom header.

Error lawmaking: DF-Delimited-ColumnDelimiterMissed

  • Message: Column delimiter is required for parse.
  • Crusade: The cavalcade delimiter is missed.
  • Recommendation: In your CSV settings, ostend that you have the column delimiter which is required for parse.

Fault code: DF-MSSQL-InvalidCredential

  • Message: Either one of user/pwd or tenant/spnId/spnKey or miServiceUri/miServiceToken should exist specified.
  • Crusade: An invalid credential is provided in the MSSQL linked service.
  • Recommendation: Please update the related MSSQL linked service with right credentials, and one of user/pwd or tenant/spnId/spnKey or miServiceUri/miServiceToken should be specified.

Error code: DF-MSSQL-InvalidDataType

  • Message: Unsupported field(s).
  • Cause: Unsupported field(southward) are provided.
  • Recommendation: Modify the input data column to match the information type supported by MSSQL.

Error lawmaking: DF-MSSQL-InvalidAuthConfiguration

  • Bulletin: Merely 1 of the iii auth methods (Key, ServicePrincipal and MI) can exist specified.
  • Cause: An invalid authentication method is provided in the MSSQL linked service.
  • Recommendation: Yous can only specify ane of the three authentication methods (Central, ServicePrincipal and MI) in the related MSSQL linked service.

Error code: DF-MSSQL-InvalidCloudType

  • Bulletin: Cloud type is invalid.
  • Cause: An invalid cloud type is provided.
  • Recommendation: Check your deject type in the related MSSQL linked service.

Error code: DF-SQLDW-InvalidBlobStagingConfiguration

  • Message: Blob storage staging properties should be specified.
  • Cause: Invalid hulk storage staging settings are provided
  • Recommendation: Delight check if the Blob linked service used for staging has correct properties.

Error lawmaking: DF-SQLDW-InvalidStorageType

  • Bulletin: Storage type can either be blob or gen2.
  • Cause: An invalid storage blazon is provided for staging.
  • Recommendation: Bank check the storage blazon of the linked service used for staging and make sure that information technology is Blob or Gen2.

Error lawmaking: DF-SQLDW-InvalidGen2StagingConfiguration

  • Message: ADLS Gen2 storage staging but back up service chief key credential.
  • Cause: An invalid credential is provided for the ADLS gen2 storage staging.
  • Recommendation: Use the service principal key credential of the Gen2 linked service used for staging.

Error code: DF-SQLDW-InvalidConfiguration

  • Message: ADLS Gen2 storage staging properties should exist specified. Either one of key or tenant/spnId/spnCredential/spnCredentialType or miServiceUri/miServiceToken is required.
  • Cause: Invalid ADLS Gen2 staging properties are provided.
  • Recommendation: Delight update ADLS Gen2 storage staging settings to have one of key or tenant/spnId/spnCredential/spnCredentialType or miServiceUri/miServiceToken.

Error code: DF-DELTA-InvalidConfiguration

  • Message: Timestamp and version tin can't exist set at the same time.
  • Cause: The timestamp and version can't be set at the aforementioned fourth dimension.
  • Recommendation: Set the timestamp or version in the delta settings.

Error code: DF-DELTA-KeyColumnMissed

  • Message: Key cavalcade(south) should be specified for non-insertable operations.
  • Crusade: Key column(southward) are missed for non-insertable operations.
  • Recommendation: Specify key column(southward) on delta sink to have not-insertable operations.

Error lawmaking: DF-DELTA-InvalidTableOperationSettings

  • Bulletin: Recreate and truncate options tin't be both specified.
  • Cause: Recreate and truncate options tin can't be specified simultaneously.
  • Recommendation: Update delta settings to have either recreate or truncate functioning.

Mistake code: DF-Excel-WorksheetConfigMissed

  • Message: Excel canvas name or index is required.
  • Cause: An invalid Excel worksheet configuration is provided.
  • Recommendation: Cheque the parameter value and specify the sheet name or index to read the Excel data.

Mistake code: DF-Excel-InvalidWorksheetConfiguration

  • Bulletin: Excel canvass proper noun and alphabetize cannot exist at the same time.
  • Cause: The Excel sheet name and index are provided at the same fourth dimension.
  • Recommendation: Check the parameter value and specify the sheet name or index to read the Excel information.

Error code: DF-Excel-InvalidRange

  • Bulletin: Invalid range is provided.
  • Cause: An invalid range is provided.
  • Recommendation: Check the parameter value and specify the valid range by the following reference: Excel format in Azure Data Manufactory-Dataset properties.

Error lawmaking: DF-Excel-WorksheetNotExist

  • Message: Excel worksheet does not be.
  • Crusade: An invalid worksheet name or index is provided.
  • Recommendation: Bank check the parameter value and specify a valid sheet name or index to read the Excel data.

Error code: DF-Excel-DifferentSchemaNotSupport

  • Bulletin: Read excel files with dissimilar schema is not supported now.
  • Cause: Reading excel files with different schemas is not supported now.
  • Recommendation: Please apply ane of post-obit options to solve this trouble:
    • Apply ForEach + data flow activity to read Excel worksheets i by i.
    • Update each worksheet schema to have the same columns manually before reading data.

Mistake code: DF-Excel-InvalidDataType

  • Bulletin: Data type is not supported.
  • Cause: The information type is not supported.
  • Recommendation: Delight change the information blazon to 'string' for related input data columns.

Fault code: DF-Excel-InvalidFile

  • Message: Invalid excel file is provided while only .xlsx and .xls are supported.
  • Crusade: Invalid Excel files are provided.
  • Recommendation: Use the wildcard to filter, and get .xls and .xlsx Excel files before reading data.

Mistake lawmaking: DF-Executor-OutOfMemorySparkBroadcastError

  • Message: Explicitly broadcasted dataset using left/right option should be modest enough to fit in node'southward memory. Yous tin can choose broadcast option 'Off' in join/exists/lookup transformation to avoid this result or utilise an integration runtime with higher memory.
  • Cause: The size of the broadcasted tabular array far exceeds the limitation of the node retentiveness.
  • Recommendation: The broadcast left/right option should be used only for smaller dataset size which can fit into node'due south retention, so brand sure to configure the node size accordingly or plough off the broadcast selection.

Error code: DF-MSSQL-InvalidFirewallSetting

  • Message: The TCP/IP connection to the host has failed. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are non blocked by a firewall.
  • Crusade: The SQL database's firewall setting blocks the data flow to access.
  • Recommendation: Delight check the firewall setting for your SQL database, and allow Azure services and resource to access this server.

Error code: DF-Executor-AcquireStorageMemoryFailed

  • Message: Transferring unroll memory to storage memory failed. Cluster ran out of memory during execution. Please retry using an integration runtime with more than cores and/or retentiveness optimized compute type.
  • Crusade: The cluster has insufficient memory.
  • Recommendation: Delight use an integration runtime with more cores and/or the retention optimized compute type.

Error code: DF-Cosmos-DeleteDataFailed

  • Message: Failed to delete data from cosmos after 3 times retry.
  • Cause: The throughput on the Cosmos collection is small and leads to coming together throttling or row data not existing in Creation.
  • Recommendation: Please accept the following actions to solve this trouble:
    • If the fault is 404, brand sure that the related row data exists in the Cosmos collection.
    • If the error is throttling, delight increment the Cosmos collection throughput or fix it to the automatic scale.
    • If the mistake is request timed out, please set up 'Batch size' in the Cosmos sink to smaller value, for example 1000.

Fault code: DF-SQLDW-ErrorRowsFound

  • Crusade: Mistake/invalid rows are found when writing to the Azure Synapse Analytics sink.
  • Recommendation: Delight find the error rows in the rejected information storage location if it is configured.

Error code: DF-SQLDW-ExportErrorRowFailed

  • Message: Exception is happened while writing error rows to storage.
  • Cause: An exception happened while writing error rows to the storage.
  • Recommendation: Please bank check your rejected information linked service configuration.

Mistake code: DF-Executor-FieldNotExist

  • Message: Field in struct does non exist.
  • Cause: Invalid or unavailable field names are used in expressions.
  • Recommendation: Cheque field names used in expressions.

Error code: DF-Xml-InvalidElement

  • Bulletin: XML Element has sub elements or attributes which can't be converted.
  • Cause: The XML element has sub elements or attributes which can't exist converted.
  • Recommendation: Update the XML file to make the XML element has right sub elements or attributes.

Error code: DF-GEN2-InvalidCloudType

  • Message: Cloud type is invalid.
  • Cause: An invalid cloud type is provided.
  • Recommendation: Bank check the cloud type in your related ADLS Gen2 linked service.

Fault code: DF-Blob-InvalidCloudType

  • Message: Cloud type is invalid.
  • Cause: An invalid cloud type is provided.
  • Recommendation: Please bank check the deject type in your related Azure Hulk linked service.

Error code: DF-Creation-FailToResetThroughput

  • Bulletin: Creation DB throughput scale operation cannot be performed because another calibration operation is in progress, please retry after sometime.
  • Cause: The throughput scale functioning of the Cosmos DB cannot be performed because another scale operation is in progress.
  • Recommendation: Please log in your Cosmos account, and manually modify its container'southward throughput to be auto scale or add custom activities after data flows to reset the throughput.

Error code: DF-Executor-InvalidPath

  • Message: Path does not resolve to any file(s). Please make certain the file/binder exists and is not hidden.
  • Cause: An invalid file/folder path is provided, which cannot exist institute or accessed.
  • Recommendation: Please cheque the file/folder path, and make sure information technology is existed and can exist accessed in your storage.

Error lawmaking: DF-Executor-InvalidPartitionFileNames

  • Message: File names cannot have empty value(s) while file name option is prepare as per partition.
  • Cause: Invalid division file names are provided.
  • Recommendation: Please check your sink settings to take the right value of file names.

Error code: DF-Executor-InvalidOutputColumns

  • Message: The result has 0 output columns. Delight ensure at least one cavalcade is mapped.
  • Crusade: No column is mapped.
  • Recommendation: Delight cheque the sink schema to ensure that at least one column is mapped.

Mistake lawmaking: DF-Executor-InvalidInputColumns

  • Message: The column in source configuration cannot exist establish in source data'south schema.
  • Cause: Invalid columns are provided on the source.
  • Recommendation: Check columns in the source configuration and make sure that information technology is the subset of the source information's schemas.

Error code: DF-AdobeIntegration-InvalidMapToFilter

  • Bulletin: Custom resource tin can only have one Cardinal/Id mapped to filter.
  • Cause: Invalid configurations are provided.
  • Recommendation: In your AdobeIntegration settings, make sure that the custom resource can merely have i Central/Id mapped to filter.

Error code: DF-AdobeIntegration-InvalidPartitionConfiguration

  • Message: Only single partition is supported. Division schema may be RoundRobin or Hash.
  • Cause: Invalid partition configurations are provided.
  • Recommendation: In AdobeIntegration settings, confirm that only the single partition is set and partition schemas may be RoundRobin or Hash.

Fault code: DF-AdobeIntegration-KeyColumnMissed

  • Message: Primal must exist specified for non-insertable operations.
  • Cause: Primal columns are missed.
  • Recommendation: Update AdobeIntegration settings to ensure key columns are specified for non-insertable operations.

Mistake code: DF-AdobeIntegration-InvalidPartitionType

  • Message: Partition type has to exist roundRobin.
  • Cause: Invalid partition types are provided.
  • Recommendation: Please update AdobeIntegration settings to make your partition type is RoundRobin.

Error code: DF-AdobeIntegration-InvalidPrivacyRegulation

  • Message: Only privacy regulation that'due south currently supported is 'GDPR'.
  • Cause: Invalid privacy configurations are provided.
  • Recommendation: Please update AdobeIntegration settings while only privacy 'GDPR' is supported.

Error code: DF-Executor-RemoteRPCClientDisassociated

  • Message: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues.
  • Cause: Data flow activeness runs failed considering of the transient network issue or because one node in spark cluster runs out of retentiveness.
  • Recommendation: Use the following options to solve this trouble:

    • Option-1: Apply a powerful cluster (both bulldoze and executor nodes have enough retentivity to handle big information) to run data flow pipelines with setting "Compute type" to "Memory optimized". The settings are shown in the motion-picture show beneath.

      Screenshot that shows the configuration of Compute type.

    • Selection-2: Use larger cluster size (for example, 48 cores) to run your data period pipelines. You lot tin acquire more almost cluster size through this document: Cluster size.

    • Option-3: Repartition your input data. For the task running on the data flow spark cluster, one partition is ane task and runs on one node. If data in one partition is also large, the related task running on the node needs to consume more memory than the node itself, which causes failure. So y'all tin can utilize repartition to avoid data skew, and ensure that data size in each sectionalisation is average while the memory consumption is not too heavy.

      Screenshot that shows the configuration of partitions.

      Notation

      You need to evaluate the data size or the division number of input data, then set reasonable partitioning number under "Optimize". For instance, the cluster that you use in the information period pipeline execution is viii cores and the retentiveness of each cadre is 20GB, only the input data is 1000GB with 10 partitions. If you lot directly run the data period, it volition meet the OOM issue because 1000GB/10 > 20GB, then information technology is ameliorate to set up repartition number to 100 (1000GB/100 < 20GB).

    • Pick-4: Melody and optimize source/sink/transformation settings. For example, endeavor to copy all files in 1 container, and don't employ the wildcard design. For more detailed information, reference Mapping data flows performance and tuning guide.

Error code: DF-MSSQL-ErrorRowsFound

  • Cause: Error/Invalid rows were establish while writing to Azure SQL Database sink.
  • Recommendation: Please detect the fault rows in the rejected data storage location if configured.

Error code: DF-MSSQL-ExportErrorRowFailed

  • Message: Exception is happened while writing error rows to storage.
  • Cause: An exception happened while writing error rows to the storage.
  • Recommendation: Check your rejected data linked service configuration.

Error code: DF-Synapse-InvalidDatabaseType

  • Message: Database type is not supported.
  • Cause: The database type is non supported.
  • Recommendation: Check the database type and change it to the proper i.

Error lawmaking: DF-Synapse-InvalidFormat

  • Message: Format is non supported.
  • Crusade: The format is not supported.
  • Recommendation: Bank check the format and change it to the proper 1.

Error code: DF-Synapse-InvalidTableDBName

  • Cause: The tabular array/database name is non valid.
  • Recommendation: Modify a valid name for the table/database. Valid names only contain alphabet characters, numbers and _.

Error code: DF-Synapse-InvalidOperation

  • Crusade: The operation is not supported.
  • Recommendation: Change the invalid operation.

Error code: DF-Synapse-DBNotExist

  • Cause: The database does not be.
  • Recommendation: Check if the database exists.

Error code: DF-Synapse-StoredProcedureNotSupported

  • Message: Employ 'Stored procedure' as Source is not supported for serverless (on-need) pool.
  • Cause: The serverless puddle has limitations.
  • Recommendation: Retry using 'query' as the source or saving the stored process as a view, and so use 'table' as the source to read from view direct.

Error code: DF-Executor-BroadcastFailure

  • Message: Dataflow execution failed during broadcast exchange. Potential causes include misconfigured connections at sources or a broadcast bring together timeout mistake. To ensure the sources are configured correctly, please examination the connexion or run a source data preview in a Dataflow debug session. To avert the broadcast join timeout, you can cull the 'Off' broadcast option in the Bring together/Exists/Lookup transformations. If you intend to use the broadcast pick to improve performance and then brand sure broadcast streams can produce data within threescore secs for debug runs and within 300 secs for task runs. If problem persists, contact customer support.

  • Cause:

    1. The source connection/configuration mistake could lead to a broadcast failure in join/exists/lookup transformations.
    2. Circulate has a default timeout of sixty seconds in debug runs and 300 seconds in chore runs. On the broadcast join, the stream chosen for the circulate seems too big to produce data within this limit. If a broadcast bring together is not used, the default circulate done by a data flow can reach the aforementioned limit.

  • Recommendation:

    • Exercise data preview at sources to confirm the sources are well configured.
    • Turn off the broadcast option or avoid broadcasting large data streams where the processing tin can have more than sixty seconds. Instead, choose a smaller stream to circulate.
    • Big SQL/Information Warehouse tables and source files are typically bad candidates.
    • In the absence of a circulate join, use a larger cluster if the fault occurs.
    • If the problem persists, contact the customer support.

Error code: DF-Cosmos-ShortTypeNotSupport

  • Message: Short data type is non supported in Cosmos DB.
  • Cause: The curt data blazon is non supported in the Azure Creation DB.
  • Recommendation: Add a derived transformation to catechumen related columns from short to integer before using them in the Cosmos sink.

Error code: DF-Blob-FunctionNotSupport

  • Message: This endpoint does not support BlobStorageEvents, SoftDelete or AutomaticSnapshot. Delight disable these business relationship features if y'all would like to utilise this endpoint.
  • Cause: Azure Blob Storage events, soft delete or automatic snapshot is not supported in data flows if the Azure Blob Storage linked service is created with service principal or managed identity authentication.
  • Recommendation: Disable Azure Blob Storage events, soft delete or automated snapshot feature on the Azure Blob account, or apply key authentication to create the linked service.

Error lawmaking: DF-Cosmos-InvalidAccountKey

  • Message: The input authorization token can't serve the asking. Please check that the expected payload is built as per the protocol, and check the key existence used.
  • Cause: There is no enough permission to read/write Azure Cosmos DB data.
  • Recommendation: Delight use the read-write key to access Azure Cosmos DB.

Next steps

For more help with troubleshooting, see these resources:

  • Mapping data flows troubleshooting guide
  • Information Mill blog
  • Data Factory characteristic requests
  • Azure videos
  • Stack Overflow forum for Data Factory
  • Twitter information most Data Mill