{"id":1287,"date":"2014-03-20T00:06:07","date_gmt":"2014-03-19T23:06:07","guid":{"rendered":"https:\/\/www.entropywins.wtf\/blog\/?p=1287"},"modified":"2014-03-20T15:21:31","modified_gmt":"2014-03-20T14:21:31","slug":"status-of-the-new-wikibase-deserialization-code","status":"publish","type":"post","link":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/","title":{"rendered":"Status of the new Wikibase (de)serialization code"},"content":{"rendered":"

A quick update on the status of the new serialization and deserialization code for Wikibase<\/a>, the software behind Wikidata<\/a>.<\/p>\n

For a long time now, we’ve had two serialization formats. One intended for external usage, and one intended for internal usage. The former one is the format our web API uses. The latter one is the format in which entities are stored in the Wikibase Repo database, and is what ends up in the current dumps.
\nThe code responsible for serialization and deserialization is for both formats not reusable and has a high degree of associated technical debt. To tackle both the reusability and design problems, we started work on two new components. One for the external format and one for the internal one.<\/p>\n

* External one: https:\/\/github.com\/wmde\/WikibaseDataModelSerialization<\/a>
\n* Internal one: https:\/\/github.com\/wmde\/WikibaseInternalSerialization<\/a><\/p>\n

They are now both essentially feature complete. The next step will be to replace usage of our existing code with usage of these new components, so the old code can be removed. Once this is done, the components will be 100% feature complete and very well tested, at which point their 1.0 versions will be released.
\nFor a description of the responsibility of both components and an explanation of how to use them, see their README files.<\/p>\n","protected":false},"excerpt":{"rendered":"

A quick update on the status of the new serialization and deserialization code for Wikibase, the software behind Wikidata. For…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[8],"tags":[67,334,181,195,197,333,317,305],"class_list":["post-1287","post","type-post","status-publish","format-standard","hentry","category-software","tag-component","tag-deserialization","tag-open-source","tag-php","tag-planet-wikimedia","tag-serialization","tag-wikibase","tag-wikidata"],"yoast_head":"\nStatus of the new Wikibase (de)serialization code - Blog of Jeroen De Dauw<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Status of the new Wikibase (de)serialization code - Blog of Jeroen De Dauw\" \/>\n<meta property=\"og:description\" content=\"A quick update on the status of the new serialization and deserialization code for Wikibase, the software behind Wikidata. For…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog of Jeroen De Dauw\" \/>\n<meta property=\"article:published_time\" content=\"2014-03-19T23:06:07+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2014-03-20T14:21:31+00:00\" \/>\n<meta name=\"author\" content=\"Jeroen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@https:\/\/twitter.com\/JeroenDeDauw\" \/>\n<meta name=\"twitter:site\" content=\"@JeroenDeDauw\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jeroen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/\"},\"author\":{\"name\":\"Jeroen\",\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/#\\\/schema\\\/person\\\/4e2ef14f2ca7dc3a0ac137d1692b66b7\"},\"headline\":\"Status of the new Wikibase (de)serialization code\",\"datePublished\":\"2014-03-19T23:06:07+00:00\",\"dateModified\":\"2014-03-20T14:21:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/\"},\"wordCount\":221,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/#\\\/schema\\\/person\\\/4e2ef14f2ca7dc3a0ac137d1692b66b7\"},\"keywords\":[\"Component\",\"deserialization\",\"Open Source\",\"PHP\",\"Planet Wikimedia\",\"serialization\",\"Wikibase\",\"wikidata\"],\"articleSection\":[\"Software\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/\",\"url\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/\",\"name\":\"Status of the new Wikibase (de)serialization code - Blog of Jeroen De Dauw\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/#website\"},\"datePublished\":\"2014-03-19T23:06:07+00:00\",\"dateModified\":\"2014-03-20T14:21:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/2014\\\/03\\\/20\\\/status-of-the-new-wikibase-deserialization-code\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Status of the new Wikibase (de)serialization code\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/\",\"name\":\"Entropy Wins\",\"description\":\"A blog on Software Architecture, Design and Craftsmanship\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/#\\\/schema\\\/person\\\/4e2ef14f2ca7dc3a0ac137d1692b66b7\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/www.entropywins.wtf\\\/blog\\\/#\\\/schema\\\/person\\\/4e2ef14f2ca7dc3a0ac137d1692b66b7\",\"name\":\"Jeroen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g\",\"caption\":\"Jeroen\"},\"logo\":{\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g\"},\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/in\\\/jeroendedauw\\\/\",\"https:\\\/\\\/x.com\\\/https:\\\/\\\/twitter.com\\\/JeroenDeDauw\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Status of the new Wikibase (de)serialization code - Blog of Jeroen De Dauw","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/","og_locale":"en_US","og_type":"article","og_title":"Status of the new Wikibase (de)serialization code - Blog of Jeroen De Dauw","og_description":"A quick update on the status of the new serialization and deserialization code for Wikibase, the software behind Wikidata. For…","og_url":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/","og_site_name":"Blog of Jeroen De Dauw","article_published_time":"2014-03-19T23:06:07+00:00","article_modified_time":"2014-03-20T14:21:31+00:00","author":"Jeroen","twitter_card":"summary_large_image","twitter_creator":"@https:\/\/twitter.com\/JeroenDeDauw","twitter_site":"@JeroenDeDauw","twitter_misc":{"Written by":"Jeroen","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/#article","isPartOf":{"@id":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/"},"author":{"name":"Jeroen","@id":"https:\/\/www.entropywins.wtf\/blog\/#\/schema\/person\/4e2ef14f2ca7dc3a0ac137d1692b66b7"},"headline":"Status of the new Wikibase (de)serialization code","datePublished":"2014-03-19T23:06:07+00:00","dateModified":"2014-03-20T14:21:31+00:00","mainEntityOfPage":{"@id":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/"},"wordCount":221,"commentCount":1,"publisher":{"@id":"https:\/\/www.entropywins.wtf\/blog\/#\/schema\/person\/4e2ef14f2ca7dc3a0ac137d1692b66b7"},"keywords":["Component","deserialization","Open Source","PHP","Planet Wikimedia","serialization","Wikibase","wikidata"],"articleSection":["Software"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/","url":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/","name":"Status of the new Wikibase (de)serialization code - Blog of Jeroen De Dauw","isPartOf":{"@id":"https:\/\/www.entropywins.wtf\/blog\/#website"},"datePublished":"2014-03-19T23:06:07+00:00","dateModified":"2014-03-20T14:21:31+00:00","breadcrumb":{"@id":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.entropywins.wtf\/blog\/2014\/03\/20\/status-of-the-new-wikibase-deserialization-code\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.entropywins.wtf\/blog\/"},{"@type":"ListItem","position":2,"name":"Status of the new Wikibase (de)serialization code"}]},{"@type":"WebSite","@id":"https:\/\/www.entropywins.wtf\/blog\/#website","url":"https:\/\/www.entropywins.wtf\/blog\/","name":"Entropy Wins","description":"A blog on Software Architecture, Design and Craftsmanship","publisher":{"@id":"https:\/\/www.entropywins.wtf\/blog\/#\/schema\/person\/4e2ef14f2ca7dc3a0ac137d1692b66b7"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.entropywins.wtf\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.entropywins.wtf\/blog\/#\/schema\/person\/4e2ef14f2ca7dc3a0ac137d1692b66b7","name":"Jeroen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g","caption":"Jeroen"},"logo":{"@id":"https:\/\/secure.gravatar.com\/avatar\/d62e6b5b8e332335cf17854fac850d9c70ba367c4692872613c3110ebd4e009b?s=96&d=mm&r=g"},"sameAs":["https:\/\/www.linkedin.com\/in\/jeroendedauw\/","https:\/\/x.com\/https:\/\/twitter.com\/JeroenDeDauw"]}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p74TBF-kL","jetpack-related-posts":[{"id":1401,"url":"https:\/\/www.entropywins.wtf\/blog\/2014\/09\/02\/wikibase-datamodel-1-0\/","url_meta":{"origin":1287,"position":0},"title":"Wikibase DataModel 1.0","author":"Jeroen","date":"2014-09-02","format":false,"excerpt":"I'm happy to announce the 1.0 release of Wikibase DataModel.\u00a0Wikibase DataModel is the canonical PHP implementation of the Data Model at the heart of the Wikibase software. This is a big release which has been some time in the making, even though many additions have been split of and included\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/www.entropywins.wtf\/blog\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1484,"url":"https:\/\/www.entropywins.wtf\/blog\/2015\/11\/08\/wikidata-wikibase-json-dump-reader\/","url_meta":{"origin":1287,"position":1},"title":"Wikidata\/Wikibase Json Dump Reader","author":"Jeroen","date":"2015-11-08","format":false,"excerpt":"I've created a small PHP library to read from, and iterate through, Wikidata\/Wikibase JSON dumps. Wikidata is the free knowledge base that anyone can edit, and serves as a central data repository for Wikipedia and associated projects. Wikibase is a set of free open source software that powers Wikidata. You\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/www.entropywins.wtf\/blog\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1327,"url":"https:\/\/www.entropywins.wtf\/blog\/2014\/04\/30\/wikibase-datamodel-entity-v2\/","url_meta":{"origin":1287,"position":2},"title":"Wikibase DataModel: Entity v2","author":"Jeroen","date":"2014-04-30","format":false,"excerpt":"In a recent blog post I introduced the new Term classes introduced in Wikibase DataModel 0.7.3. It also outlined plans for making some big changes to the Entity class and it's derivatives. We have now taken the most difficult step in the process, which is already resulting in much nicer\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/www.entropywins.wtf\/blog\/category\/programming\/"},"img":{"alt_text":"dm-complexity","src":"https:\/\/i0.wp.com\/www.entropywins.wtf\/blog\/wp-content\/uploads\/2014\/04\/dm-complexity.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1453,"url":"https:\/\/www.entropywins.wtf\/blog\/2015\/08\/11\/wikibase-datamodel-services\/","url_meta":{"origin":1287,"position":3},"title":"Wikibase DataModel Services","author":"Jeroen","date":"2015-08-11","format":false,"excerpt":"I'm happy to announce the immediate availability of a new Wikibase library: Wikibase DataModel Services (which I'll in this blog post refer to as DMS). Rationale behind the library The main motivation for introducing this new library is to reduce technical debt and draw more solid architectural boundaries in the\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/www.entropywins.wtf\/blog\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1524,"url":"https:\/\/www.entropywins.wtf\/blog\/2016\/01\/25\/replicator-a-cli-tool-for-wikidata\/","url_meta":{"origin":1287,"position":4},"title":"Replicator: Wikidata import tool","author":"Jeroen","date":"2016-01-25","format":false,"excerpt":"I'm happy to announce the first release of Replicator, a CLI tool for importing entities from Wikidata. Replicator was created for importing data from Wikidata into the QueryR REST API persistence. It has two big conceptual components: getting entities from a specified source, and then doing something with said entities.\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/www.entropywins.wtf\/blog\/category\/programming\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1165,"url":"https:\/\/www.entropywins.wtf\/blog\/2013\/12\/23\/wikibase-datamodel-released\/","url_meta":{"origin":1287,"position":5},"title":"Wikibase DataModel released!","author":"Jeroen","date":"2013-12-23","format":false,"excerpt":"I\u2019m happy to announce the 0.6 release of Wikibase DataModel. This is the first real release of this component. DataModel? Wikibase is the software behind Wikidata.org. At its core, this software is about describing entities. Entities are collections of claims, which can have qualifiers, references and values of various different\u2026","rel":"","context":"In \"Component\"","block_context":{"text":"Component","link":"https:\/\/www.entropywins.wtf\/blog\/tag\/component\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/posts\/1287","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/comments?post=1287"}],"version-history":[{"count":4,"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/posts\/1287\/revisions"}],"predecessor-version":[{"id":1291,"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/posts\/1287\/revisions\/1291"}],"wp:attachment":[{"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/media?parent=1287"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/categories?post=1287"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.entropywins.wtf\/blog\/wp-json\/wp\/v2\/tags?post=1287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}