I can’t read file contents from my Android SD card on Linux and it’s weird

Leave a reply

I want to backup my Android SD card so I inserted it to my computer (tried multiple ones by now). I can see the file tree and browse it just fine. But if I try to read a file, it fails because instead of the actual file contents, all I get is some gibberish. Weirdly enough, the file header is partially identical between different files and contains the words “whatever” and “CONSOLE”. When inserted in my phone I can read and use those files just fine. The phone is a Sony Xperia XZ1 Compact.

Here are some example hexdumps of the file headers from some JPG and OPUS files:

00000000  00 00 00 00 00 37 d5 23  d8 c3 44 86 e4 42 f3 73  |.....7.#..D..B.s|
00000010  03 00 00 02 00 00 10 00  00 02 8c 2d 04 09 03 01  |...........-....|
00000020  77 68 61 74 65 76 65 72  60 ed 60 a5 16 dd d0 08  |whatever`.`.....|
00000030  34 0d 25 2b 87 d1 df 18  94 8a f8 cf f0 fd 83 d9  |4.%+............|
00000040  06 5a 4e 48 8c a1 b9 51  98 ed 16 62 08 5f 43 4f  |.ZNH...Q...b._CO|
00000050  4e 53 4f 4c 45 00 00 00  00 60 40 2d b1 41 51 c1  |NSOLE....`@-.AQ.|
00000060  10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000  c7 27 3f d6 7e 99 3d 6b  d3 dc 0a a7 c9 28 37 d5  |.'?.~.=k.....(7.|
00002010  59 09 8b 4b 3e 1b 20 54  a8 87 fc 90 fd 31 05 5b  |Y..K>. T.....1.[|
...
0037ffe0  f1 9a 2d fa fd a1 4b 2a  22 dc ce 29 9d 83 3a 5a  |..-...K*"..)..:Z|
0037fff0  dc 97 e0 e9 15 d7 16 55  82 a6 57 6b 7f b0 32 7d  |.......U..Wk..2}|
00380000

00000000  00 00 00 00 00 03 5d 1d  aa ab 3d 85 96 2a 8a 70  |......]...=..*.p|
00000010  03 00 00 02 00 00 10 00  00 02 8c 2d 04 09 03 01  |...........-....|
00000020  77 68 61 74 65 76 65 72  60 90 17 d4 8b ef 31 35  |whatever`.....15|
00000030  22 b9 b3 05 59 37 3b 71  7e e1 4a 6e af a2 07 b2  |"...Y7;q~.Jn....|
00000040  4b 9a bb 7e 6a 46 18 70  29 ed 16 62 08 5f 43 4f  |K..~jF.p)..b._CO|
00000050  4e 53 4f 4c 45 00 00 00  00 60 40 2d b1 41 51 c1  |NSOLE....`@-.AQ.|
00000060  10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000  32 23 5d 69 cb 14 d0 75  8a 9c 2e 64 67 26 1a fc  |2#]i...u...dg&..|
00002010  12 77 ad 48 9b f4 9e d4  d5 08 48 33 cb 45 6b 90  |.w.H......H3.Ek.|
...
00037fe0  87 09 0c 80 6c 4b 6a 54  fb c2 70 13 bc 0a c9 ab  |....lKjT..p.....|
00037ff0  89 3b 52 40 f2 cc df f4  d3 65 e1 c4 0d e3 74 ea  |.;R@.....e....t.|
00038000

00000000  00 00 00 00 00 01 78 41  33 26 6e 9c 0f a7 d9 69  |......xA3&n....i|
00000010  03 00 00 02 00 00 10 00  00 02 8c 2d 04 09 03 01  |...........-....|
00000020  77 68 61 74 65 76 65 72  60 2e 2d e8 1b cd 47 11  |whatever`.-...G.|
00000030  7e b6 e7 e2 95 84 85 75  81 42 0f 5c 54 48 3e 4b  |~......u.B.\TH>K|
00000040  c7 3c f6 cf 16 ec 9f 6b  51 ed 16 62 08 5f 43 4f  |.<.....kQ..b._CO|
00000050  4e 53 4f 4c 45 00 00 00  00 60 40 2d b1 41 51 c1  |NSOLE....`@-.AQ.|
00000060  10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000  97 9d 44 91 25 8e b4 e0  07 31 31 bc cc c0 44 13  |..D.%....11...D.|
00002010  62 0e 6d c1 49 8d 87 fe  5d c7 ff c0 cc b5 4c 08  |b.m.I...].....L.|
...
00019fe0  ae 2b d1 04 db d2 9c 9e  e7 cd 3a 06 aa 5a 85 5f  |.+........:..Z._|
00019ff0  d3 7a 30 33 f2 3a b1 2f  73 40 50 f7 a9 a6 d7 fa  |.z03.:./s@P.....|
0001a000

fdisk shows the partition as “W95 FAT32 (LBA)“.

What’s going on?

I found some mentions of this issue on the web:

https://old.reddit.com/r/techsupport/comments/f88bd7/files_on_sd_card_taken_out_of_phone_wont_open/ posted “iu>i'Õè Œ- whatever`P‘Ã@C“Žâ=ë q¸ÌÔ9rÊ ¯Ø¶Þ¨Kíb_CONSOLE ]ŽÒ[Ó_” without a solution.

https://www.computerbase.de/forum/threads/videodatei-samsung-a-3-2016-am-pc-abspielen.1737550/ posted “@Âù©õš•Bo   Œ-  `»Üž¸½\ht1Oc.ä9á²5Ñé’Ž<žŸ9Ûßíb_CONSOLE Øþ«fQ¿É“, also without a solution. The phone involved was a Samsung A3 2016.

https://www.nikon-fotografie.de/community/threads/jpg-dateien-pl%C3%B6tztlich-nicht-mehr-lesbar.281280/ posted “#µ=Äõ¶EBC   Œ-  "3DUfw`â￾Lü4OOŸ-›·j.^aÌ?–ôaÚ ”YÜíb_CONSOLE .øŒ¤Èœ«é ‰'ñÏÆísÿä‰â¢THû´/Ò†´< bb·7iìÕ“, also without a solution.

https://superuser.com/questions/975695/reading-encrypted-files-from-android-microsd-card posted three examples, all also having at least the “b._CONSOLE” portion, Samsung Galaxy S3 device. No solution but a user commenting that “This is related to Samsung Knox and these signatures are Samsung specific” which seems unlikely as I am using a Sony device…

Und tschüß, Forum/TPHH-Spiegel community.codeforhamburg.org

Leave a reply

community.codeforhamburg.org war ein (auf Basis von Discourse aufgesetztes) Forum für Code for Hamburg e. V., online vom Mai 2019 bis April 2023.

Die Homepage
Jeder Eintrag im Transparenzportal Hamburg wurde gespielt
Man konnte Suchen speichern und bei neuen Ergebnissen eine Mail erhalten
Ein Beispiel für einen Eintrag
Ein Eintrag mit mehreren verknüpften Resourcen

Die Idee war ursprünglich, dass dort Kommentare und Diskussionen zu den im Transparenzportal Hamburg (TPHH) veröffentlichten Datensätzen gepostet werden könnten. Und das vielleicht sogar direkt im TPHH eingebettet.

Dazu hat ein Python-Skript (als Bot namens “Heidi Kabel” ;) ) jede Minute die API des TPHH nach neuen Datensätzen gefragt, die Metadaten abgerufen und sie schön formatiert und aufbereitet als Foren-Threads gepostet. Ein spaßiges Bastel-Projekt, irgendwann 2019.

Die Dokumentation für die Forumsinstallation und der Code für den Bot liegen in:

Naja, dann kam Corona und Treffen von Code for Hamburg gab’s eh schon kaum noch. Die Seite hat praktisch niemand außer mir genutzt, es gab keine 10 Posts, die nicht vom Bot waren. Mit der Einbindung im TPHH wurde es auch nichts (ich hatte mich aber auch nicht drum gekümmert).

“Angesagte” Posts mit vielen Views
Pageviews exklusive (erkannte) Crawler
Top Referred Topics (keine Ahnung wo sie verlinkt wurden)
Der tägliche User war unser TPHH-Spiegel-Bot, die Posts/Topics auch

Ich hatte es für mich selbst noch als einfaches Interface für die Daten und als praktischen Benachrichtigungsservice auf bestimmte Keywords genutzt, aber jetzt klappt das Update der Forensoftware nicht mehr (Bug von Discourse mit angeblich nicht mehr einzigartigen Tags und es gab keine Rückmeldung auf meine Fragen, von daher …) und ich will da nix auf veralteter Software laufen lassen. Übernehmen wollte es niemand aus dem Verein und im Fediverse hat sich auch niemand gemeldet. Von daher mach ich die Tage das Licht aus.

PS: Ich hoffe Code for Hamburg wird irgendwann mal wieder reaktiviert, mit regelmäßigen Treffen, wo gehackt und gebastelt wird. Toi toi toi!

10 year old posts from an older blog at enjoys.it

Leave a reply

Just for save-keeping, probably irrelevant nowadays but who knows what kind of travelers search engines might send here. Tread lightly!

Remove the huge margins from PDF papers

Remove the huge margins from PDF papers eg. prior to printing with pdfcrop from the pdftk suite. For example:

pdfcrop --margins -80 input.pdf output.pdf

edit, much better:

$ pdfimages -j file.pdf file
$ mogrify -crop 800x1300+220+220 +repage file-0*.jpg
$ convert file-0*.jpg cropped.pdf

+repage makes IM not write the offset to the files but actually crop “properly”. Maybe pdfcrop has such parameter too?

Installing a printer in Cups via a USB->Serial adapter

Make sure to “modprobe usbserial“. dmesg should show a printer being connected if you plug it in now. Then Cups should see it too.

RTTY with SDR# and fldigi (for the german DWD stations)

In SDR#: Use USB, filter bandwidth 1000. center the RTTY in your window.

In fldigi: Op Mode -> RTTY -> Custom. Set the carrier shift to custom and then enter 450 in the custom shift field below. Baud rate: 50, 5 bits per character, no parity, 1.5 stop bits. Save and Close. Make sure the Rv button is green!

CW decoding with SDR# and fldigi

In SDR#: Use the CW-L or CW-U preset. Tune so the morse code is right in the middle of your “reception window”.

In fldigi: Op Mode -> CW. Turn off squelch by making the SQL button not green but grey.

It works very well for non-human morse for me. Radio amateur morse is harder and so far full of “spelling errors”. :)

PDF to image with imagemagick/graphicsmagick

If you want to create images from PDF files, use for example mogrify -verbose -geometry 1600 -density 300 -format png *.pdf Without a decent “-density” parameter, you will probably get a blurry image as result.

Making your Ryzen CPU less hot by throttling boost on Linux

Leave a reply

echo 0 | tee /sys/devices/system/cpu/cpufreq/boost

Might work, might not, depending on unknown factors. Whatever, I just wanted to make some all-core-but-unimportant process to run without going 95°C. For that it worked perfectly well. CPU temps of a Ryzen 5 3600 after many hours of full utilization were at ~65°C. CPU frequencies were capped to 3.6GHz with this while jumping up to 4.2GHz (and ~94°C) without.

Obviously this has an impact on performance.

To re-enable just echo a 1 instead. This is reset anyways when you reboot your system.

Properly setting up your QGIS license

1 Reply

If you want your copy of QGIS display it’s legal licensing status, this is the missing code for you.

Copy this in your QGIS Python script editor (WARNING: DO NOT RUN THIS IN AN IMPORTANT USER PROFILE, I will NOT help you if it breaks something):

import os
from qgis.core import QgsApplication, QgsSettings
from qgis.PyQt.QtGui import QColor, QImage, QPainter, QPen, QStaticText
from qgis.PyQt.QtWidgets import QMessageBox

def install_qgis_license():
    # WARNING: This fucks around with your profiles and stuff!
    # QgsCustomization is not available from Python so just yolo here and write a fresh file if possible
    profile_directory = QgsApplication.qgisSettingsDirPath()
    customization_ini_filepath = profile_directory + "QGIS/QGISCUSTOMIZATION3.ini"

    if os.path.isfile(customization_ini_filepath):
        # ain't gonna be touchin dat!
        text = (
            "QGISCUSTOMIZATION3.ini EXISTS! UNLICENSED HACKING DETECTED!\n"
            "Or: Custom license has been installed already...\n"
            "Anyways, we are not creating a *new* license now ;)"
        )
        messagebox = QMessageBox()
        messagebox.setText(text);
        messagebox.exec()
        return

    # get existing splash image
    splash_path = QgsApplication.splashPath()  # :/images/splash/
    splash_image_file = splash_path + "splash.png"
    splash_image = QImage(splash_image_file)

    # paint new splash image
    new_splash_image = QImage(splash_image)
    painter = QPainter()
    painter.begin(new_splash_image)

    # white bold font plz
    font = QgsApplication.font()
    font.setBold(True)
    painter.setFont(font)
    pen = painter.pen()
    pen.setColor(QColor("white"))
    painter.setPen(pen)

    # place text at appropriate location
    label_text = f"This QGIS©®™ is legally licensed to {os.getlogin()}"
    label_text_rect = painter.boundingRect(0, 0, 0, 0, 0, label_text)  # returns new rect that fits text
    left_offset = new_splash_image.width() - label_text_rect.size().width() - 20
    painter.drawText(left_offset, 840, label_text)

    painter.end()

    # create license dir if necessary
    new_splash_dir_path = profile_directory + "license"
    try:
        os.mkdir(new_splash_dir_path)
    except FileExistsError:
        pass

    save_success = new_splash_image.save(new_splash_dir_path + "/splash.png")
    if save_success:
        print(f"Initialized new QGIS license....")
    else:
        print("Error on QGIS license initialization, this will get reported!")
        return

    # enable license dir for splash image lookup in QGISCUSTOMIZATION3.ini
    with open(customization_ini_filepath, "w") as sink:
        sink.write("[Customization]\n")
        sink.write(f"splashpath={new_splash_dir_path}/")  # must have trailing slash

    # enable loading of QGISCUSTOMIZATION3.ini in user profile
    QgsSettings().setValue(r"UI/Customization/enabled", True)
    
    print("License installed, reboot QGIS to activate!")
    messagebox = QMessageBox()
    messagebox.setText("License installed, restart QGIS now to activate!");
    messagebox.exec()

#install_qgis_license()

Then (if you really want to do it), uncomment the function call in the last line and execute the script. Follow the instructions.

To clean up remove or restore the QGIS/QGISCUSTOMIZATION3.ini file in your profile and remove the license directory from your profile, restore the previous value of UI/Customization/enabled in your profile (just remove the line or disable Settings -> Interface Customization).

If you want to hate yourself in the future, put it in a file called startup.py in QStandardPaths.standardLocations(QStandardPaths.AppDataLocation) aka the directory which contains the profiles directory itself.

BTW: If you end up with QGIS crashing and lines like these in the error output:

...
Warning: QPaintDevice: Cannot destroy paint device that is being painted
QGIS died on signal 11
...

It is probably not a Qt issue that caused the crash. The QPaintDevice warning might just be Qt telling you about your painter being an issue during clean up of the actual crash (which might just be a wrong name or indentation somewhere in your code, cough).

Using a Tiardey USB Single Foot Pedal (PCsensor FootSwitch) on Linux

Leave a reply

This post’s purpose is to link “Tiardey USB Single Foot Pedal Optical Switch Control One Key Programm Computer Tastatur Maus Game Action HID” to “PCsensor” and the footswitch tool on search engines so others who wonder if the device is easy to use on Linux learn that this is the case. Hope it helps!

I bought this https://www.amazon.de/dp/B09TQFBS3C which came with a chinese/manual saying “FS2007 User Manual” and also says “FS2007U1SW (mechanical switch)” (mine clicks, so I guess it is not the “FS2007U1IR (silent photoelectric switch)”. The manual links to pcsensor.com for Windows drivers.

Plugin the device. dmesg should show something like:

[Sun Jan 8 20:25:05 2023] usb 1-4: new full-speed USB device number 7 using xhci_hcd
[Sun Jan 8 20:25:05 2023] usb 1-4: New USB device found, idVendor=1a86, idProduct=e026, bcdDevice= 0.00
[Sun Jan 8 20:25:05 2023] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[Sun Jan 8 20:25:05 2023] usb 1-4: Product: FootSwitch
[Sun Jan 8 20:25:05 2023] usb 1-4: Manufacturer: PCsensor
[Sun Jan 8 20:25:06 2023] input: PCsensor FootSwitch Keyboard as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.0/0003:1A86:E026.0001/input/input19
[Sun Jan 8 20:25:06 2023] input: PCsensor FootSwitch Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.0/0003:1A86:E026.0001/input/input20
[Sun Jan 8 20:25:06 2023] input: PCsensor FootSwitch as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.0/0003:1A86:E026.0001/input/input21
[Sun Jan 8 20:25:06 2023] hid-generic 0003:1A86:E026.0001: input,hidraw0: USB HID v1.11 Keyboard [PCsensor FootSwitch] on usb-0000:00:14.0-4/input0
[Sun Jan 8 20:25:06 2023] input: PCsensor FootSwitch as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.1/0003:1A86:E026.0002/input/input22
[Sun Jan 8 20:25:06 2023] hid-generic 0003:1A86:E026.0002: input,hidraw1: USB HID v1.10 Device [PCsensor FootSwitch] on usb-0000:00:14.0-4/input1
[Sun Jan 8 20:25:06 2023] usbcore: registered new interface driver usbhid
[Sun Jan 8 20:25:06 2023] usbhid: USB HID core driver
[Sun Jan 8 20:25:10 2023] usb 1-4: reset full-speed USB device number 7 using xhci_hcd

Sweet, so it is just some rebranded PCsensor device.

lsusb says ID 1a86:e026 QinHeng Electronics FootSwitch btw.

There is a great little tool for configuring those on Linux: https://github.com/rgerganov/footswitch

footswitch -m ctrl -k 1 will configure it to send Ctrl+1 when pressed for example. See the readme for usage and more examples.

You can use more than 3 of these devices via this pull request. I have four connected via a USB hub 1a40:0101 (“Terminus Technology Inc. Hub (branded “hama”), https://www.amazon.de/dp/B08YRZT1RL) and they work just fine.

Animated snowflakes in QGIS

Leave a reply

Inspired by https://mapstodon.space/@bogind/109558968869019746 and listening to https://www.youtube.com/watch?v=AdyAF9M3XVw on repeat thanks to https://ruby.social/@halfbyte/109558803178815394 I made:

This was the final expression (with lots of opportunity to improve):

with_variable(
  'point_at_top_of_canvas',
  densify_by_count(
    make_line(
      point_n( @map_extent, 3),  -- no idea if these indexes are stable
      point_n( @map_extent, 4)
    ),
    42  -- number of trajectories
  ),
  collect_geometries(
    array_foreach(
      generate_series(1, num_points(@point_at_top_of_canvas)),
      with_variable(
        'point_n_of_top_line',
        point_n(@point_at_top_of_canvas, @element),
        point_n(
          wave_randomized(
            make_line(
              @point_n_of_top_line,
              -- make it at least touch the bottom of the canvas:
              translate(@point_n_of_top_line, 0, -@map_extent_height)
            ),
            -- fairly stupid frequency and wavelength but hey, works in any crs
            1, @map_extent_width/5,
            1, @map_extent_width/100,
            seed:=@element  -- stable waves \o/
          ),
          floor(epoch(now())%10000/50)  -- TODO make it loop around according to num_points of each line
        )
      )
    )
  )
)

Use it on an empty polygon layer with an inverted polygon style and set it to refresh at a high interval (0.01s?). Or use this QGIS project (I included some intermediate steps of the style as layer styles if you want to learn about this kind of stuff):

20221222-animated-snowflakes.qgz_Download

Welcome to #GISmaster

Leave a reply

“Social media” was a mistake so I am bringing over some nice things I had posted on Twitter to this blog.

I love Taskmaster and … well:

  • Bring the most surprising sidecar file.
  • Recreate a OSGeo project’s logo using these materials, best recreation wins.
  • Trigger error 99999 as many times as you can. Most different reasons wins.
  • Bring the best value to find in a date column.
  • Georeference this monochrome scan of a hand-drawn 1910 map, most accurate wins.
  • Draw a world map. Most area conformal map wins.
  • Name as many QGIS releases as you can, most release names wins.
  • Bring the best typeface for a remote village.
  • Invent a new Shapefile sidecar file. Most useful sidecar file wins.
  • Make the most beautiful choropleth map. You have 5 minutes to name your data sources then 15 minutes to make your map.
  • Bring the most disturbing data preparation story.
  • Demonstrate a standard GIS operation as interpretative dance.
  • Get a social media leech to repost a map of yours without credit. Most retweets wins.
  • Bring the worst communal geo data portal.
  • Create the coolest hex grid map from this dataset of beehives.
  • Spell/pronounce GDAL’s main developer’s name correctly.
  • Imitate Ian Turton when a demanding user calls him on his private number for GeoServer support.
  • Bring a sane GML file.
  • Write a standard-compliant CityGML parser.
  • Design the most beautiful north arrow.
  • Choose an angle. (…) (…) Design a tube map only using lines at the angle you chose earlier.

Kate contributed “Determine what a Shapefile projection should be without a .prj file or metadata” :D

Previously at https://twitter.com/cartocalypse/status/1530251866000465920

#30DayMapChallenge as #1Day30MapsChallenge (2021)

2 Replies

Not sure why I never posted this last year but I did the #30DayMapChallenge in a single day, streamed live via a self-hosted Owncast instance. It was … insane and fun. This year I will do it again, on the 26th of November.

Here are most of the maps I made last year:

Some notes I kept, please bug me about recovering the others from my Twitter archive (I deleted old tweets a bit too early):

  • 1 Points: Pins via Geometry Generator in QGIS
  • 2 Lines: River elevation profiles of Elbe, Rhein, Ems, Weser, Donau and Main. DEM: © GeoBasis-DE / BKG (2021)
  • 13 NaEr I mean Natural Earth (Blame @tjukanov)
  • 18 Water (DGM-W 2010 Unter- und Außenelbe, Wasserstraßen- und Schifffahrtsverwaltung des Bundes, http://kuestendaten.de, 2010)
  • 20 Movement: Emojitions on a curvy trajectory. State changes depending on the curvyness ahead. Background: (C) OpenStreetMap Contributors <3
  • 21 Elevation with qgis2threejs (It’s art, I swear!
  • 22 Boundaries: Inspired by Command and Conquer Red Alert. Background by Spiney (CC-BY 3.0 / CC-BY-SA 3.0, https://opengameart.org/node/12098)
  • 24 Historical: Buildings in Hamburg that were built before the war (at least to some not so great dataset). Data Lizenz: Datenlizenz Deutschland Namensnennung 2.0 (Freie und Hansestadt Hamburg, Landesbetrieb Geoinformation und Vermessung (LGV))
  • 27 Heatmap: Outdoor advertisements (or something like that) in Hamburg. Fuck everything about that! Data Lizenz: Datenlizenz Deutschland Namensnennung 2.0 (Freie und Hansestadt Hamburg, Behörde für Verkehr und Mobilitätswende, (BVM))
  • 28 Earth not flat. Using my colleague’s Beeline plugin to create lines between the airports I have flown too and the Globe Builder plugin by @gispofinland to make a globe.