`RawObjectDataProcessor` — Demo

Documentation Page has not been checked by Native Speakers yet

We are sorry, this this page has not been checked by English native speakers yet.

Suppose the value of the parameter externalData is obtained from an external data source that is unreachable for type checking by TypeScript.

function onDataRetrieved(externalData: unknown): void {
  // ...
}

In particular, the following scenarios are possible:

Data retrieving from the client side during client–server interaction
Conversely, data retrieving from the server side during client–server interaction
Retrieving data from a database
Reading data from a file (JSON, YAML, and similar)

We expect that the received data will have the type SampleType, however, what these data will actually be at runtime is unknown at advance to us at the code-writing stage; therefore, the parameter has the type unknown, and not SampleType.

type SampleType = {
  foo: number;
  bar: string;
  baz: boolean;
  hoge?: number;
  fuga: string | null;
  quux: {
    alpha: number;
    bravo: "PLATINUM" | "GOLD" | "SILVER";
  };
};

Because of this, simply forcing the type like externalData as SampleType is unsafe. Although due to the nature of TypeScript you will have to do it eventually (see "problem overview"), this must be backed up by something — namely, by validation.

Solving this problem with RawObjectDataProcessor, it is necessary to define a valid data specification — essentially, an object that describes metadata for specific data, in particular the expected types of each property, their requirements, and other limitations. For demonstration, in addition to specifying the expected types of the properties, some other restrictions are added as well, for example, the maximal characters count for string properties or a list of possible values. In practice, such validation is in considerable demand, so it will not make this example excessive.

const validDataSpecification: RawObjectDataProcessor.ObjectDataSpecification = {
  nameForLogging: "Example",
  subtype: RawObjectDataProcessor.ObjectSubtypes.fixedSchema,
  properties: {
    foo: {
      type: Number,
      isUndefinedForbidden: true,
      isNullForbidden: true,
      numbersSet: RawObjectDataProcessor.NumbersSets.positiveIntegerOrZero,
      isNaN_Forbidden: true
    },
    bar: {
      type: String,
      isUndefinedForbidden: true,
      isNullForbidden: true,
      minimalCharactersCount: 5
    },
    baz: {
      type: Boolean,
      isUndefinedForbidden: true,
      isNullForbidden: true
    },
    hoge: {
      type: Number,
      isUndefinedForbidden: false,
      isNullForbidden: true,
      isNaN_Forbidden: true,
      numbersSet: RawObjectDataProcessor.NumbersSets.positiveIntegerOrZero
    },
    fuga: {
      type: Number,
      isUndefinedForbidden: true,
      isNullForbidden: false,
      numbersSet: RawObjectDataProcessor.NumbersSets.positiveIntegerOrZero,
      isNaN_Forbidden: true
    },
    quux: {
      type: Object,
      isUndefinedForbidden: true,
      isNullForbidden: true,
      properties: {
        alpha: {
          type: Number,
          isUndefinedForbidden: true,
          isNullForbidden: true,
          numbersSet: RawObjectDataProcessor.NumbersSets.anyInteger,
          isNaN_Forbidden: true,
          minimalValue: 3
        },
        bravo: {
          type: String,
          isUndefinedForbidden: true,
          isNullForbidden: true,
          minimalCharactersCount: 5,
          allowedAlternatives: [ "PLATINUM", "GOLD", "SILVER" ]
        }
      }
    }
  }
};

When the valid data specification is defined, pass it as a parameter to the static method process of the class RawObjectDataProcessor together with the raw input data still of the type unknown:

function onDataRetrieved(externalData: unknown): void {

  const externalDataProcessingResult: RawObjectDataProcessor.ProcessingResult<SampleType> = RawObjectDataProcessor.
      process(externalData, validDataSpecification);

}

The value returned by the method process has the type defined as follows:

export type ProcessingResult<ProcessedData> =
    Readonly<
      {
        isRawDataInvalid: false;
        processedData: ProcessedData;
      } |
      {
        isRawDataInvalid: true;
        validationErrorsMessages: ReadonlyArray<string>;
      }
    >;

A peculiarity here is that we cannot access the property processedData until we will prove to TypeScript that isRawDataInvalid property is falsy:

const externalDataProcessingResult: RawObjectDataProcessor.ProcessingResult<SampleType> = RawObjectDataProcessor.
    process(externalData, validDataSpecification);

console.log(externalDataProcessingResult.processedData);

Accessing externalDataProcessingResult.processedData without first checking externalDataProcessingResult.isRawDataInvalid sans violating TypeScript is impossible — this will cause the error “TS2339: Property processedData does not exist on type ProcessingResult<SampleType>”. The error message is imprecise because it is not that the type ProcessingResult<SampleType> as a whole lacks the property, but rather that one of its subtypes does — specifically the subtype with isRawDataInvalid: true. Accordingly, before accessing the property processedData, you need to exclude the subtype that lacks this property. This can be done with a regular conditional statement:

if (externalDataProcessingResult.isRawDataInvalid) {
  console.log(externalDataProcessingResult.validationErrorsMessages);
}

But when isRawDataInvalid is true, it is still impossible to access processedData property because it simply does not exist; instead, you can access the array that contains validation error messages — that is, descriptions of all mismatches between the actual data and the expected one. How to react to these validation errors depends on the specifics of your task, but usually it is notifying the user and/or throwing an error:

if (externalDataProcessingResult.isRawDataInvalid) {

  throw new InvalidExternalDataError({
    mentionToExpectedData: "N External Data",
    messageSpecificPart: RawObjectDataProcessor.
        formatValidationErrorsList(externalDataProcessingResult.validationErrorsMessages)
  });

}

As seen in the example above, RawObjectDataProcessor also provides a static method for formatting of validation error messages. For example, when calling the function onDataRetrieved with the following input data:

onDataRetrieved({
  foo: -4,
  bar: "abc",
  quux: {
    alpha: 2,
    bravo: "BRONZE"
  }
});

Node.js will output the following log:

InvalidExternalDataError: The data "N External Data" does not match with expected.
─── Error No. 1 ────────────────────────────────────────────────────────────────
Expected and Actual Numbers Set Mismatch [ VALIDATION_ERRORS_MESSAGES-NUMERIC_VALUE_IS_NOT_BELONG_TO_EXPECTED_NUMBERS_SET ]

● Property / Element: foo
Contrary to expectations, this numeric value is not member of "positive integer or zero"
See documentation for details: https://ee.yamato-daiwa.com/CoreLibrary/Functionality/RawObjectDataProcessor/Children/06-ValidationIssues/RawObjectDataProcessor-ValidationIssues.english.html#VALIDATION_ERRORS_MESSAGES-NUMERIC_VALUE_IS_NOT_BELONG_TO_EXPECTED_NUMBERS_SET

●　Property / Element Specification:
{
  type: "number",
  isUndefinedForbidden: true,
  isNullForbidden: true,
  numbersSet: "POSITIVE_INTEGER_OR_ZERO",
  isNaN_Forbidden: true
}
● Actual Value: -4

─── Error No. 2 ────────────────────────────────────────────────────────────────
Minimal Characters Count Fall Short [ VALIDATION_ERRORS_MESSAGES-CHARACTERS_COUNT_IS_LESS_THAN_REQUIRED ]

● Property / Element: bar
This string has 3 characters while at least 5 required.
See documentation for details: https://ee.yamato-daiwa.com/CoreLibrary/Functionality/RawObjectDataProcessor/Children/06-ValidationIssues/RawObjectDataProcessor-ValidationIssues.english.html#VALIDATION_ERRORS_MESSAGES-CHARACTERS_COUNT_IS_LESS_THAN_REQUIRED

●　Property / Element Specification:
{
  type: "string",
  isUndefinedForbidden: true,
  isNullForbidden: true,
  minimalCharactersCount: 5
}
● Actual Value: "abc"

─── Error No. 3 ────────────────────────────────────────────────────────────────
Forbidden Undefined Value [ VALIDATION_ERRORS_MESSAGES-FORBIDDEN_UNDEFINED_VALUE ]

● Property / Element: baz
This property/element is not defined or have explicit `undefined` value what has been explicitly forbidden.
See documentation for details: https://ee.yamato-daiwa.com/CoreLibrary/Functionality/RawObjectDataProcessor/Children/06-ValidationIssues/RawObjectDataProcessor-ValidationIssues.english.html#VALIDATION_ERRORS_MESSAGES-FORBIDDEN_UNDEFINED_VALUE

●　Property / Element Specification:
{
  type: "boolean",
  isUndefinedForbidden: true,
  isNullForbidden: true
}
● Actual Value: undefined

─── Error No. 4 ────────────────────────────────────────────────────────────────
Forbidden Undefined Value [ VALIDATION_ERRORS_MESSAGES-FORBIDDEN_UNDEFINED_VALUE ]

● Property / Element: fuga
This property/element is not defined or have explicit `undefined` value what has been explicitly forbidden.
See documentation for details: https://ee.yamato-daiwa.com/CoreLibrary/Functionality/RawObjectDataProcessor/Children/06-ValidationIssues/RawObjectDataProcessor-ValidationIssues.english.html#VALIDATION_ERRORS_MESSAGES-FORBIDDEN_UNDEFINED_VALUE

●　Property / Element Specification:
{
  type: "number",
  isUndefinedForbidden: true,
  isNullForbidden: false,
  numbersSet: "POSITIVE_INTEGER_OR_ZERO",
  isNaN_Forbidden: true
}
● Actual Value: undefined

─── Error No. 5 ────────────────────────────────────────────────────────────────
Minimal Value Fall Short [ VALIDATION_ERRORS_MESSAGES-NUMERIC_VALUE_IS_SMALLER_THAN_REQUIRED_MINIMUM ]

● Property / Element: quux.alpha
This value is smaller than required minimal value 3.
See documentation for details: https://ee.yamato-daiwa.com/CoreLibrary/Functionality/RawObjectDataProcessor/Children/06-ValidationIssues/RawObjectDataProcessor-ValidationIssues.english.html#VALIDATION_ERRORS_MESSAGES-NUMERIC_VALUE_IS_SMALLER_THAN_REQUIRED_MINIMUM

●　Property / Element Specification:
{
  type: "number",
  isUndefinedForbidden: true,
  isNullForbidden: true,
  numbersSet: "ANY_INTEGER",
  isNaN_Forbidden: true,
  minimalValue: 3
}
● Actual Value: 2

─── Error No. 6 ────────────────────────────────────────────────────────────────
Disallowed Alternative of Value [ VALIDATION_ERRORS_MESSAGES-VALUE_IS_NOT_AMONG_ALLOWED_ALTERNATIVES ]

● Property / Element: quux.bravo
This value is not among following allowed alternatives:
  ○ PLATINUM
  ○ GOLD
  ○ SILVER
See documentation for details: https://ee.yamato-daiwa.com/CoreLibrary/Functionality/RawObjectDataProcessor/Children/06-ValidationIssues/RawObjectDataProcessor-ValidationIssues.english.html#VALIDATION_ERRORS_MESSAGES-VALUE_IS_NOT_AMONG_ALLOWED_ALTERNATIVES

●　Property / Element Specification:
{
  type: "string",
  isUndefinedForbidden: true,
  isNullForbidden: true,
  minimalCharactersCount: 5,
  allowedAlternatives: [
    "PLATINUM",
    "GOLD",
    "SILVER"
  ]
}
● Actual Value: "BRONZE"
    at onDataRetrieved (D:\Demo.ts:87:11)
    at <anonymous> (D:\Demo.ts:99:1)
    at Object.<anonymous> (D:\Demo.ts:106:2)
    at Module._compile (node:internal/modules/cjs/loader:1730:14)
    at Object.transformer (C:\Users\I\AppData\Roaming\npm\node_modules\tsx\dist\register-DpmFHar1.cjs:2:1186)
    at Module.load (node:internal/modules/cjs/loader:1465:32)
    at Function._load (node:internal/modules/cjs/loader:1282:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:235:24)
    at cjsLoader (node:internal/modules/esm/translators:266:5)

Node.js v22.15.0

If isInvalid is false, then you can access the property processedData and use it:

if (externalDataProcessingResult.isRawDataInvalid) {

  throw new InvalidExternalDataError({
    mentionToExpectedData: "N External Data",
    messageSpecificPart: RawObjectDataProcessor.
        formatValidationErrorsList(externalDataProcessingResult.validationErrorsMessages)
  });

}


console.log(externalDataProcessingResult.processedData)

Here you can already access the property processedData, because the subtype { isRawDataInvalid: true; validationErrorsMessages: ReadonlyArray<string> } has been excluded by throwing an error inside the preceding if-block. If the function or method that contains the code above is not required to return valid data, then instead of throwing an error you can use the keyword return.

Besides fixed-structure objects as in the example above, RawObjectDataProcessor can also validate other kinds of objects:

Associative arrays — differ from fixed-structure objects by an arbitrary number of key–value pairs; however, all values must follow a single pattern.
Indexed arrays
Tuples — differ from indexed arrays by a fixed number of elements and their order; moreover, each element may have specific constraints defined.

Why is the class named RawObjectDataProcessor and not RawObjectDataValidator? Because it can not only validate data but also transform it — in particular, rename keys and change values. All this and much more is described further in this documentation.

RawObjectDataProcessor — Demo

`RawObjectDataProcessor` — Demo