{"id":667182,"date":"2020-07-20T09:40:46","date_gmt":"2020-07-20T16:40:46","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=667182"},"modified":"2022-04-19T15:09:26","modified_gmt":"2022-04-19T22:09:26","slug":"prose-json-transformation","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/prose-json-transformation\/","title":{"rendered":"PROSE &#8211; Json Transformation"},"content":{"rendered":"<p><strong>Transformation.Json<\/strong> transforms the structure of Json using input\/output examples.<\/p>\n<p>The Usage page and the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/microsoft\/prose\/tree\/main\/api-samples\/Transformation.Json\">Sample Project<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> illustrate the API usage.<\/p>\n<h2>Example Transformation<\/h2>\n<p>Given an example to transform this input Json:<\/p>\n<pre class=\"brush: plain; auto-links: true; title: ; notranslate\" title=\"\">\r\n{\r\n  \"datatype\": \"local\",\r\n  \"data\": &#x5B;\r\n    {\r\n      \"Name\": \"John\",\r\n      \"status\": \"To Be Processed\",\r\n      \"LastUpdatedDate\": \"2013-05-31 08:40:55.0\"\r\n    },\r\n    {\r\n      \"Name\": \"Paul\",\r\n      \"status\": \"To Be Processed\",\r\n      \"LastUpdatedDate\": \"2013-06-02 16:03:00.0\"\r\n    }\r\n  ]\r\n}\r\n<\/pre>\n<p>into this output Json:<\/p>\n<pre class=\"brush: plain; auto-links: true; title: ; notranslate\" title=\"\">\r\n&#x5B;\r\n  {\r\n    \"John\" : \"To Be Processed\"\r\n  },\r\n  {\r\n    \"Paul\" : \"To Be Processed\"\r\n  }\r\n]\r\n<\/pre>\n<p><strong>Transformation.Json<\/strong> will generate a program to perform the same transformation given any other similar and larger input Json. For example, the learned program transforms this input:<\/p>\n<pre class=\"brush: plain; auto-links: true; title: ; notranslate\" title=\"\">\r\n{\r\n  \"datatype\": \"local\",\r\n  \"data\": &#x5B;\r\n    {\r\n      \"Name\": \"John\",\r\n      \"status\": \"To Be Processed\",\r\n      \"LastUpdatedDate\": \"2013-05-31 08:40:55.0\"\r\n    },\r\n    {\r\n      \"Name\": \"Paul\",\r\n      \"status\": \"To Be Processed\",\r\n      \"LastUpdatedDate\": \"2013-06-02 16:03:00.0\"\r\n    },\r\n    {\r\n      \"Name\": \"Alice\",\r\n      \"status\": \"Finished\",\r\n      \"LastUpdatedDate\": \"2013-07-02 12:04:00.0\"\r\n    }\r\n  ]\r\n}\r\n<\/pre>\n<p>into this output:<\/p>\n<pre class=\"brush: plain; auto-links: true; title: ; notranslate\" title=\"\">\r\n&#x5B;\r\n  {\r\n    \"John\" : \"To Be Processed\"\r\n  }, \r\n  { \r\n    \"Paul\" : \"To Be Processed\"\r\n  }, \r\n  { \r\n    \"Alice\" : \"Finished\"\r\n  } \r\n]\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Transformation.Json transforms the structure of Json using input\/output examples.<\/p>\n","protected":false},"featured_media":674232,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13554,13560],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-667182","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-human-computer-interaction","msr-research-area-programming-languages-software-engineering","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[],"related-downloads":[],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[{"id":0,"name":"Usage","content":"The main entry point is <code class=\"language-plaintext highlighter-rouge\">Session<\/code> class\u2019s <code class=\"language-plaintext highlighter-rouge\">Learn()<\/code> method, which returns a <code class=\"language-plaintext highlighter-rouge\">Program<\/code> object. The <code class=\"language-plaintext highlighter-rouge\">Program<\/code>\u2019s key method is <code class=\"language-plaintext highlighter-rouge\">Run()<\/code> that executes the program on an input Json to obtain the transformed output Json.\r\n\r\nOther important methods are <code class=\"language-plaintext highlighter-rouge\">Serialize()<\/code> and <code class=\"language-plaintext highlighter-rouge\">Deserialize()<\/code> to serialize and deserialize <code class=\"language-plaintext highlighter-rouge\">Program<\/code> object.\r\n\r\nThe <a href=\"https:\/\/github.com\/microsoft\/prose\/tree\/main\/api-samples\/Transformation.Json\">Sample Project<\/a> illustrates our API usage.\r\n<h2>Basic Usage<\/h2>\r\nThe following snippet illustrates a learning session to generate a Json transformation program from the example:\r\n\r\n[code autolinks=\"true\" highlight=\"\"]\r\n\/\/ The training input file, which is a small prefix of the input file.\r\nJToken trainInput = JToken.Parse(@\"\r\n{\r\n  \"\"datatype\"\": \"\"local\"\",\r\n  \"\"data\"\": [\r\n    {\r\n      \"\"Name\"\": \"\"John\"\",\r\n      \"\"status\"\": \"\"To Be Processed\"\",\r\n      \"\"LastUpdatedDate\"\": \"\"2013-05-31 08:40:55.0\"\"\r\n    },\r\n    {\r\n      \"\"Name\"\": \"\"Paul\"\",\r\n      \"\"status\"\": \"\"To Be Processed\"\",\r\n      \"\"LastUpdatedDate\"\": \"\"2013-06-02 16:03:00.0\"\"\r\n    }\r\n  ]\r\n}\");\r\n\r\n\/\/ The training output file, which is the desired output for the training input.\r\nJToken trainOutput = JToken.Parse(@\"\r\n[\r\n    {\r\n      \"\"John\"\" : \"\"To Be Processed\"\"\r\n    },\r\n    {\r\n      \"\"Paul\"\" : \"\"To Be Processed\"\"\r\n    }\r\n]\");\r\n\r\nvar session = new Session();\r\nsession.Constraints.Add(new Example(trainInput, trainOutput));\r\nProgram program = session.Learn();[\/code]\r\n\r\n<div style=\"height: 30px\"><\/div>\r\n<h2>Serializing\/Deserializing a Program<\/h2>\r\nThe <code class=\"language-plaintext highlighter-rouge\">Transformation.Json.Program.Serialize()<\/code> method serializes the learned program to a string.\r\nThe <code class=\"language-plaintext highlighter-rouge\">Transformation.Json.Loader.Instance.Load()<\/code> method deserializes the program text to a program.\r\n\r\n[code autolinks=\"true\" highlight=\"\"]\r\n\/\/ program was learned previously\r\nstring progText = program.Serialize();\r\nProgram loadProg = Loader.Instance.Load(progText);[\/code]\r\n\r\n<div style=\"height: 30px\"><\/div>\r\n<h2>Executing a Program<\/h2>\r\nUse the <code class=\"language-plaintext highlighter-rouge\">Run()<\/code> method to obtain the transformed Json output:\r\n\r\n[code autolinks=\"true\" highlight=\"\"]\r\nJToken input = JToken.Parse(@\"\r\n{\r\n  \"\"datatype\"\": \"\"local\"\",\r\n  \"\"data\"\": [\r\n    {\r\n      \"\"Name\"\": \"\"John\"\",\r\n      \"\"status\"\": \"\"To Be Processed\"\",\r\n      \"\"LastUpdatedDate\"\": \"\"2013-05-31 08:40:55.0\"\"\r\n    },\r\n    {\r\n      \"\"Name\"\": \"\"Paul\"\",\r\n      \"\"status\"\": \"\"To Be Processed\"\",\r\n      \"\"LastUpdatedDate\"\": \"\"2013-06-02 16:03:00.0\"\"\r\n    },\r\n    {\r\n      \"\"Name\"\": \"\"Alice\"\",\r\n      \"\"status\"\": \"\"Finished\"\",\r\n      \"\"LastUpdatedDate\"\": \"\"2013-07-02 12:04:00.0\"\"\r\n    }\r\n  ]\r\n}\");\r\n\r\nJToken output = program.Run(input);[\/code]\r\n\r\nOutput is:\r\n\r\n[code autolinks=\"true\" highlight=\"\"]\r\n[\r\n  {\r\n    \"John\" : \"To Be Processed\"\r\n  },\r\n  {\r\n    \"Paul\" : \"To Be Processed\"\r\n  },\r\n  {\r\n    \"Alice\" : \"Finished\"\r\n  }\r\n][\/code]\r\n"}],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Vu Le","user_id":39174,"people_section":"Section name 0","alias":"levu"}],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/667182"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":4,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/667182\/revisions"}],"predecessor-version":[{"id":789851,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/667182\/revisions\/789851"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/674232"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=667182"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=667182"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=667182"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=667182"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=667182"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}