{"id":1246,"date":"2019-04-24T14:37:44","date_gmt":"2019-04-24T12:37:44","guid":{"rendered":"https:\/\/hannes.enjoys.it\/blog\/?p=1246"},"modified":"2019-05-09T10:54:11","modified_gmt":"2019-05-09T08:54:11","slug":"i-tried-to-package-a-packagecligui-for-python","status":"publish","type":"post","link":"https:\/\/hannes.enjoys.it\/blog\/2019\/04\/i-tried-to-package-a-packagecligui-for-python\/","title":{"rendered":"I tried to package a package+CLI+GUI for Python"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">And I gave up trying.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here is our project: <a href=\"https:\/\/gitlab.com\/g2lab\/cogran-python\">https:\/\/gitlab.com\/g2lab\/cogran-python<\/a><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>There is a python package in <code>cogran\/<\/code>.<\/li><li>There are scripts in <code>scripts\/<\/code>, one CLI interface, one QT GUI.<\/li><li>There are docs in <code>docs\/<\/code>.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The GUI uses images and text from the docs for live help.<br><strong>So we need to make sure that our <\/strong><code><strong>docs<\/strong><\/code><strong> directory is included when packaging this all for others to install.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I tried for two days to make sense of the jungle of Python packaging. StackOverflow is full of non-explanatory half-answers, there are about 7 APIs implemented by 35 different projects with 42 different documentations and 98 different versions. Either I massively failed to find the one, wonderful, canonical, documented approach or this really is neither explicit nor beautiful.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So I give up. Can <strong>you<\/strong> help me out?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Do we need to integrate the docs into the actual package? Should we instead have three separate things: The cogran module package, a cogran-cli package, a cogran-gui package?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The setup<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">For building I used <br><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>rm -r build\/ dist\/ cogran.egg-info\/ ; \\\npython setup.py sdist &amp;&amp; \\\npython setup.py bdist_wheel<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">For looking into the resulting archives I used<br><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>watch tar tvf dist\/cogran-0.1.0.tar.gz<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">and<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>watch unzip -t dist\/cogran-0.1.0-py3-none-any.whl<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">For installing I used<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip uninstall --yes cogran ; \\\npip install --no-cache-dir --user --upgrade dist\/cogran-0.1.0.tar.gz &amp;&amp; \\\nfind ~\/.local\/ | grep cogran<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">and<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip uninstall --yes cogran ; \\\npip install --no-cache-dir --user --upgrade dist\/cogran-0.1.0-py3-none-any.whl &amp;&amp; \\\nfind ~\/.local\/ | grep cogran<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">What happens<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">sdist includes all kinds of stuff like the .gitignore and the examples directory. Both should not be included at the moment. Installing the resulting cogran-0.1.0.tar.gz will just install the package (cogran\/__init__.py) and the scripts. Nothing else.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">bdist_wheel only includes the package and the scripts. Nothing else.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Adding a MANIFEST.in<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">After fucking around with many iterations of fruitless attempts of writing a MANIFEST.in file that beautifully builds upon the someone-said default includes I gave in and wrote a very explicit one: https:\/\/gitlab.com\/snippets\/1850708<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>global-exclude *\n\ngraft cogran\ngraft docs\ngraft scripts\ngraft tests\n\ninclude README.md\ninclude LICENSE\n\ninclude setup.py\ninclude requirements.txt\n\nglobal-exclude __pycache__\nglobal-exclude *.py[co]<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Now sdist includes all the stuff I want and nothing I do not want. Excellent. Installing the tar.gz however again only installed the package and the scripts.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The wheel again only had the package and the scripts inside.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">setup.py: include_package_data=True<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You just need to add include_package_data=True to your setup.py, people said. So I did. And no, that would only work for things inside our package, not directories on the same level. So this would change nothing.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">setup.py: data_files<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Supply the paths to the files in a data_files list, someone said. So I added an explicit list (at this stage I just wanted it to work, who cares about manual effort, future breakage and ugliness):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>data_files = [\n  (\"docs\", (\n    \"docs\/images\/Areal Weighting.png\",\n    \"docs\/images\/Attribute Weighting.png\",\n    \"docs\/images\/postcodes vs districts.png\",\n    \"docs\/images\/Binary Dasymetric Weighting.png\",\n    \"docs\/images\/N-Class Dasymetric Weighting.png\",\n    \"docs\/help.html\"\n  ))\n]<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And yes, now these files end up in the wheel. But guess what, they still do not get installed anywhere<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">setup.py: package_data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/pypa\/sampleproject\/blob\/master\/setup.py#L164\">https:\/\/github.com\/pypa\/sampleproject\/blob\/master\/setup.py#L164<\/a> like include_package_data only applies to data <em>inside<\/em> the package, our&#8217;s is on the side.<br><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What now?<\/h3>\n","protected":false},"excerpt":{"rendered":"<p>And I gave up trying. Here is our project: https:\/\/gitlab.com\/g2lab\/cogran-python There is a python package in cogran\/. There are scripts in scripts\/, one CLI interface, one QT GUI. There are docs in docs\/. The GUI uses images and text from the docs for live help.So we need to make sure that our docs directory is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43],"tags":[],"class_list":["post-1246","post","type-post","status-publish","format-standard","hentry","category-python"],"_links":{"self":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts\/1246","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/comments?post=1246"}],"version-history":[{"count":10,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts\/1246\/revisions"}],"predecessor-version":[{"id":1259,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/posts\/1246\/revisions\/1259"}],"wp:attachment":[{"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/media?parent=1246"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/categories?post=1246"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hannes.enjoys.it\/blog\/wp-json\/wp\/v2\/tags?post=1246"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}