How do I set special parameters in an AWS Glue job using AWS CloudFormation?
"I want to enable special parameters, such as --enable-metrics, for my job in AWS Glue. However, I get a template validation or "null values" error from AWS CloudFormation when I try to run my job. How do I resolve these errors?"
"I want to enable special parameters, such as --enable-metrics, for my job in AWS Glue. However, I get a template validation or "null values" error from AWS CloudFormation when I try to run my job. How do I resolve these errors?Short descriptionTo set special parameters for your job in AWS Glue, you must supply a key-value pair for the DefaultArguments property of the AWS::Glue::Job resource in CloudFormation. If you supply a key only in your job definition, then CloudFormation returns a validation error.Resolution1.    In your CloudFormation template, set the value of your special parameter to an empty string for the DefaultArguments property of your job definition.JSON:"MyJob": {  "Type": "AWS::Glue::Job",  "Properties": {    "Command": {      "Name": "glueetl",      "ScriptLocation": "s3://my-test//test-job1"    },    "DefaultArguments": {      "--job-bookmark-option": "job-bookmark-enable",      "--enable-metrics": ""    },    "ExecutionProperty": {      "MaxConcurrentRuns": 2    },    "MaxRetries": 0,    "Name": "cf-job3",    "Role": {      "Ref": "MyJobRole"    }  }}YAML:MyJob:  Type: 'AWS::Glue::Job'  Properties:    Command:      Name: glueetl      ScriptLocation: 's3://my-test//test-job1'    DefaultArguments:      '--job-bookmark-option': job-bookmark-enable      '--enable-metrics': ''    ExecutionProperty:      MaxConcurrentRuns:     MaxRetries: 0    Name: cf-job3    Role: !Ref MyJobRoleNote: In the preceding example JSON and YAML templates, the value of --enable-metrics is set to an empty string. The empty string validates the template and launches the resource that's configured with the special parameter.2.    To activate your special parameter, run your job.Follow"