Rosetta Translate

Rosetta Translate

Rosetta Translate enables the user to keep in control of a mapping layer between their existing systems, databases and message formats and the format underpinned by a model. The main artefact of the mapping layer is called the Translation Dictionary, which is created thanks to Rosetta Translate.

Rosetta Translate is closely linked to another Rosetta product: Rosetta Ingest. Rosetta Ingest offers an universal API to ingest and normalise the user data, by leveraging on the mapping layer that is created thanks to Rosetta Translate.

In this section, you will learn about:

  • Ingestion and Projection
  • Exploring the file
  • Uploading a file
  • Running Ingestion
  • Viewing the results

This section uses examples extracted from the Demonstration Model, a model based on vehicle taxonomy which is used for tutorial and training. Users are invited to refer to the Mapping Section of the Rosetta DSL documentation for further details on how mappings are implemented.

Ingestion and Projection

In Rosetta, Ingestion is the process of converting an (electronic) input file into the format underpinned by a model.

The input file can be one of different standard formats for electronic data storage and transport, such as XML or JSON.

The model output can be:

  • An electronic file, using the same range of support formats as the input file (a.k.a. a Serialised model document)
  • A model object that computer code can be directly executed on (a.k.a. a De-Serialised model object)

The Serialised view is useful to display the model output into an interface, in a way that a user can browse through and understand. This is the view that is used when demonstrating how Rosetta Translate really works. The De-Serialised object is not meant to be readable.

Conversely, Projection is the process of converting a model object into an (electronic) file with the targeted format.

Exploring the files

The File Explorer allows the user to interact with sample files which are specific to the loaded Workspace.

translate_ingestion_explorer_overview

File Explorer overview

Filter Bar

The Filter Bar provides the ability to limit which files are displayed.

The Filter Bar is made up of the following components:

  • Synonym Sources: Use this drop down to view only sample files that run against a particular synonym source
  • Show Results: Checking the checkbox will only show sample files whose tests have been run and have results to display
  • File Name: Enter text here to filter by sample files whose name contains that text
  • Run Button: Select to run all files in the current filtered view, this button also shows the number of results

File Upload

The Upload button allows users to select sample files from their desktop and store them either on the server or in their browser on their computer. The decision to store on the server or not depends on whether the Save In My Workspace checkbox beneath the Upload button is ticked.

Files uploaded to the browser on the users computer will be removed on logout

File Explorer View

The File Explorer View is split up into Group Rows and Sample File rows.

Group Rows

The Group Row hierarchy structure is dictated by the location of the sample files within the model and their purpose is a logical categorisation of the various sample files.

The Group Row is made up of five components:

  • Group Name: Sample file group name.
  • Synonym Source: The oval synonym source marker shows which synonym sources are used by each of the sample files in that group.
  • Group Status: The group status shows how many files have been run in this group.
  • Group Completeness: These three pie charts show the individual mapping, validation and qualification success percentages.
  • Run All Action: Runs all sample files within the group.

Sample File Rows

The Sample File Rows allow the user to interact with each sample file associated with one group, trigger test runs, view run summary information and navigate to the detailed analysis page.

The Sample File Row is made up of the following components:

  • Diagnostic Summary: This summary shows a pie chart with the percentage success calculated against an expected value for each of the following:
    • Mapping: These are are the number of paths in the source file that have mapped successfully to paths in the selected model.
    • Validation: This result shows what percentage of validation rules passed for the qualified type.
    • Qualification: A qualification is the type that an ingested source is assigned having passed a set of criteria in a qualification function.
  • Detailed Diagnostic Summary: When hovering over each of the diagnostic summary charts described above, each of these popup summaries contains the following:
    • Success: The actual vs expected success count for that result type.
    • Failure: The actual vs expected failure count for that result type.
    • Excluded: The actual vs expected excluded count for that result type.

  • Rerun: The rerun buttons appear when the results have gone stale due to a model change and allow the user to refresh the results.

  • File Run States: Here are all the row states:
    • Pending rows will have a grey background
    • Successful rows will have a green background, all their expectations have passed
    • Failed rows will have a red background and mean that the file has failed to pass the expected Ingestion tests
    • Warning rows will have a yellow background and mean that an error occurred when running the Ingestion
    • Stale rows will have a grey background
Sample Files available in the Sample File Rows are specific to each selected model

Uploaded File Rows

Uploaded files have a slightly different view. An uploaded file has Diagnostic Summary which only displays actual counts for Mapping and Validation. There are no expectations to compare the results against for these file types, unlike for Sample File Rows. The user can still interact with these rows to run and rerun in the same way as Sample File Rows.

The Uploaded File Row contains the following components:

  • Selected Synonym Source: This displays the synonym source a file will be ingested against
  • File location icon: Files which have been saved in the Workspace display the cloud icon

Uploading a File

When uploading an electronic file for ingestion, use the upload button. Select the file of interest, choose the synonym source as needed, and determine the storage location, either within your workspace or on your device.

Opting to store the file in your workspace offers the advantage of accessibility across different computers and login sessions. This ensures that the file remains available seamlessly.

The file needs to be physically accessible from the user's computer. Only supported files will be selectable. Ingestion in Rosetta currently supports XML and JSON input types.

Running Ingestion

Provided that all the required code in the user Workspace is ready to be run, the user can process one or multiple files through Ingestion by doing one of the following:

  • Single File Run: Clicking on a Sample File Row will start the process of Ingestion, and a spinner will appear at the right end of the row
  • Group File Run: Clicking on the Run Group button on a Group Row will start processing all sample files in that Group, and a spinner will appear next to all files subject to Ingestion
  • Filtered Run: After applying a filter, the Run all files button will appear with a number of results. Clicking this button will start processing all results.
  • Rerunning Files: If a file has not been processed or results have gone stale, the user can reprocess a file by clicking the rerun button. By contrast, a Group can be reprocessed at anytime by clicking on the Run Group button.

Viewing the results

When a file has results, the user can click on a Sample File Row to open the Result Viewer.

Result Viewer

The Result Viewer displays the results for a given sample file. It is split vertically into three panels, from left to right:

  • Diagnostics panel: this pannel displays statistics (Mapping, Validation, Qualification) and other key information summarising the success (or failure) of ingesting the file.
  • Input File View and Model Output View panels: these panels allow the user to review how the system has ingested the input file. The Input File View corresponds to the input file used for Ingestion and the Model Output View corresponds to the output in the model format.

Each panel can be opened and closed by clicking on its header

View Modes

The Input File View and the Model Output View have two view modes:

  • Code View: displays the file in its original mark-up format
  • Formatted Document View: creates a tree structure which is colour coded to indicate the result of the Ingestion process

The Formatted Document View uses a list of colours to give mapping information between the input file and the model output

  • Red: Invalid or unmapped values
  • Dark Green: Mapped values
  • Dark Green With underline: Mapped and linked values from input file to model output
  • Light Green: Conditional values
  • Yellow: Excluded values
  • Black: No mapping data for these values
Clicking on a dark green field which is underlined will highlight and scroll into view the corresponding fields in the input file or model output Code View