{"id":1723,"date":"2022-03-13T11:09:37","date_gmt":"2022-03-13T10:09:37","guid":{"rendered":"https:\/\/hannes.enjoys.it\/blog\/?p=1723"},"modified":"2023-08-15T14:08:16","modified_gmt":"2023-08-15T12:08:16","slug":"transparenzportal-hamburg-api-alle-daten-herunterladen","status":"publish","type":"post","link":"https:\/\/hannes.enjoys.it\/blog\/2022\/03\/transparenzportal-hamburg-api-alle-daten-herunterladen\/","title":{"rendered":"Transparenzportal Hamburg API: Alle Daten herunterladen"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">https:\/\/suche.transparenz.hamburg.de.EXAMPLE.COM\/api\/action\/resource_search?query=url: (&#8220;<code>.EXAMPLE.COM<\/code>&#8221; entfernen) liefert aktuell rund 200 Megabyte an JSON, da sollten alle Resourcen drin stecken oder zumindest die, die tats\u00e4chlich einen Datensatz referenzieren<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Um es in normalen Editoren besser handlebar zu machen, hilft <code>json_pp<\/code>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">cat resource_search\\?query=url\\: | json_pp &gt; url\\:.json_pp<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Und die URLs bekommt man (<a href=\"https:\/\/hannes.enjoys.it\/blog\/2016\/10\/transparenzportal-hamburg-api-alle-datensatze-eines-bestimmten-hosts\/\">wie in meinem alten Post schon<\/a>) mit<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">grep '\"url\"' url\\:.json_pp | grep -Eo 'http.*\"' | sed 's#\"$##' &gt; urls<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Oder die Gesamtgr\u00f6\u00dfe via <code><a href=\"https:\/\/stackoverflow.com\/a\/451204\">paste | bc<\/a><\/code>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">$ grep '\"file_size\"' url\\:.json_pp | grep -Po \"\\d+\" | paste -s -d+ - | bc\n\n3751796542539<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Rund 4 Terabyte? Schauen wir mal.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/suche.transparenz.hamburg.de.EXAMPLE.COM\/api\/action\/resource_search?query=url: (&#8220;.EXAMPLE.COM&#8221; entfernen) liefert aktuell rund 200 Megabyte an JSON, da sollten alle Resourcen drin stecken oder zumindest die, die tats\u00e4chlich einen Datensatz referenzieren Um es in normalen Editoren besser handlebar zu machen, hilft json_pp: Und die URLs bekommt man (wie in meinem alten Post schon) mit Oder die Gesamtgr\u00f6\u00dfe via paste | bc: Rund [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[32,27,10,28],"tags":[],"class_list":["post-1723","post","type-post","status-publish","format-standard","hentry","category-german","category-hamburg","category-open-data","category-transparenz"],"_links":{"self":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts\/1723","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/comments?post=1723"}],"version-history":[{"count":5,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts\/1723\/revisions"}],"predecessor-version":[{"id":1969,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts\/1723\/revisions\/1969"}],"wp:attachment":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/media?parent=1723"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/categories?post=1723"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/tags?post=1723"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}