When attempting to ingest a fixed length file, I am seeing records dropping and I am unsure why.
A basic example of my file where the 2nd record will be dropped:
The 2nd record in the above example is being dropped due to not fitting the criteria of a fixed length record.
Padding is required in order to maintain proper formatting. In the event of a string, padding will be added on the tail end in the form of additional spaces. If the data were an integer, the padding would appear on the head end in the form of zeroes.
In order to properly ingest the above second record, it would have to be repaired in the following way:
We have added the additional tailing spaces for the first column and front loaded zeroes on the integer in the second column.