FME's Exceptional Null Handling

5. Readers and Writers  Readers now read Nulls  Oracle, SQL Server, CSV, Access etc  Writers now write Nulls and Update to Null  Oracle, SQL Server, Geodatabase etc  Creators now make Nulls  AttributeCreator  AttributeValueMapper

6. Wørking with Nulls  NullAttributeMapper  Change nulls to values  Change values to nulls  SQLCreator/Executor  Correctly reads attribute schema – no more missing attributes

7. Wørking with Nulls  AttributeClassifier  Now tests for nulls  Tester – in all its forms  Now differentiates null/missing/empty

8. Stay Up-to-date with Nøthing  FMEpedia.com http://fme.ly/xyn fme.ly/nulls How does FME handle null attribute values?  Be aware of known issues  Still ongoing to solve even more arcane issues  Information regarding format and transformer support for nothing

9. Thank Yøu!  Questions?  For more information:  info@safe.com  www.safe.com

Notas do Editor

Much ado about nothing: Benedick and Beatrice (the main couple) Null & Missing , Claudio and Hero (the secondary couple) not compatibleAbout 15 different ways to represent nothing in databases, XML (nil), raster etc.Introduce the concepts of “Value based nodata”, missing, empty and nothing “Value based” nodata– i.e. MapInfo -9999 for null integers of floats. Raster nodata value (often RGB 0,0,0)Missing – the attribute doesn’t existEmpty – the value of the attribute is <blank> which is ASCII characterNull – the value of the attribute is <null> which is ASCII character NUL hex:0In SQL compliant databases <null> represents the ‘unknown’ valueIn FME 2013 <null> was represented as the ‘missing’ attribute, which conflicts with database writers. Char vs. varchar : char(10) is padded with <blank> charactersvarchar(10) can have up to 10 characters. Empty varchar is <null>
FME 2014 – major core work done to support the concept of nulls. Up until now we have always just treated attributes with null values as missing attributes. So to implement null support involved touching just about every transformer in the product and many of the readers and writers.So – why – nulls are meaningful to our database customers. Correctly identifying that an attribute has a null value is more “correct” than just assuming that the attribute is missing or the string is empty. Why do you care? – let’s see what changes were made alongside adding null support that will benefit all users.
Data Inspector – now shows what is really in your attribute. Prior to FME 2013 you could not distinguish between an attribute with an empty string or a blank value. Attributes would be missing from the attribute list if they had actually held null values rather than actually being missingNow we show <null> if there really is nulls in an attribute – this is in the Feature Information window and in the table viewerWe also show <missing> if the format is not a null aware format (e.g. mapinfo) and the attribute really is missingWe also do the same thing in the logger. This all more correctly identifies what is actually in your data
Prior to FME 2014 our writers did usually write null values in the database formats because at the point of writing the database libraries take over. However it was on the Readers that we were dropping attributes if they had null values. Major format readers now correctly pass attributes with null values into the workspaceWriters continue to write null values to output formats and in addition can update existing attribute values to null as long as the incoming attribute has a value of null. No longer a need to fake out the writerCreator transformers can be used to create attributes with null values, attributes can be set to null values, and mapped to null values.
New transformers – NullAttributeMapper – replaces the NullAttributeReplacer. Can now completely manage what you want to do with null, missing or empty attributes. Even allows for mapping attribute values from one value to another. Works on multiple attributes.Aliased as the BulkAttributeMapper.SQLCreator and SQLExecutor now expose all attributes to the workspace even if they have null values. This used to be a major problem with these transformers as you were likely to have to manually expose a large number of attributes in these transformers – this is now handled correctly.
Miscellaneous transformers were also modified to provide extra value. AttributeClassifier – can filter attributes based on if they have a null value. Null <> numeric <> stringTester in all it’s forms - now differentiates null/missing/empty
Check out this fmepedia page for more specific information regarding which formats we support and any current issues. We are continuing this work and in 2015 beta we have fixed the ExpressionEvaluator to handle nulls.Work is ongoing and we are interested in any feedback.Links are to FMEpedia and iMark’s blog.

FME's Exceptional Null Handling

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de Safe Software

Mais de Safe Software (20)

Último

Último (20)

FME's Exceptional Null Handling

Notas do Editor