A question came up in private conversations, but the answer IMHO is useful generally, so this is a repost. The question was generally on the topic of divercity and outreach - if it does really help underpriviledged people to give them some handout and hope that that somehow turns them into priviledged people? IMHO this is an important question in current world environment in general and to Debian in particular.

IMHO It's not about making unprivileged people more privileged than current contributors or about giving some kind of handouts.

Just talking from my personal perspective, it is more about analysing where past decisions (even perfectly reasonable ones) inadvertently created biases and trying to figure out ways to balance them out over time so that we don't end up missing good contributors just because they were, for example, unlucky with the country of their birth.

A simple example - think back a couple decades - where was the best place in the world to learn how to develop free software in general and Debian in particular? Where you had the largest chances of meeting an experienced developer in person with enough free time who could show you the ropes and explain to you things that are hard to put into documentation? I'd say those would be the campuses of a bunch of technical universities in the USA, UK and the rest of Western Europe.

If you were not attending a university, you were out of luck. If you were rural in a smaller campus, you were out of luck. If you were on a different continent - guess what? Still out of luck. Even Eastern Europe - so close, and yet - out of luck.

This made perfect sense and was a perfectly reasonable way to operate back then - Western universities had the highest density of people who had both the skill and the free time to contribute to free software. It was a no-brainer to start there. It was not a mistake to do so. But it had side-effects.

I was very lucky and persistent to overcome this out-of-luck situation, despite the closest developers being 500-600 km away and across a sea and me operating on actual dial-up Internet with barely enough money for a bus ticket, never mind a sea ferry or a plane ticket. Or a laptop to take on such a plane. Thousands of others around me, including many very competent software developers, did not have that kind of luck and persistence and thus are not contributing to free software or Debian at all.

If we look back at the past, we can recognise that past decisions left whole continents and also whole classes of people that were not given the opportunity to learn how and why to contribute to free software and Debian. That is in the end a realisation that there are many fine future developers out there who we have not reached out to yet. At this point that is a great untapped source of new developers for the project. Opportunities in western universities will still be there, noone wants to or can tear them down, but they also do not really need investment of attention, time and money to keep being effective. But other places do need that investment to start being effective. To start adding new developers to the community.

Outreach to diverse communities is in our, selfish interests. Naturally we still need to figure out how much effort is actually spent on that and evaluate efficiency of different approaches. But in principle it is what we want to do.

(And also our knowledge might make their lives better, but that's a minor side effect. /s)

Questions? Feedback? Just ask here or here.

Two years of midsize electric sedan experience

This February (2026) marks a full 10 years since I started working for BMW, and a key employment bonus is the ability to drive a company car on special two-year leasing terms. Just before the new year 2026 started, I said goodbye to my latest company car. After driving the BMW i4, I switched to the BMW i5. In terms of power, it was a downgrade as I switched from the maximum power i4 M50 xDrive (all-wheel drive, 600 hp) to the i5 eDrive40 (rear-wheel drive, 340 hp). Did I regret that? Not for a single second! After driving 60,000 km in the last two years with the BMW i5, I was really sad to let it go -- it was the best car I have ever driven. Simple as that.

Comparing i5 sDrive40 to i4 M50 xDrive

First, about the drivetrain. Technically, these two cars had the same generation of the BMW electrical drivetrain and very similar driving and charging characteristics. One had two electric motors (one on the rear axle and one on the front axle) and the other had just one motor on the rear axle. This also causes the difference in motor power -- basically, the power of the front motor is added on top of the power that the rear motor provides. There are motors with different power outputs, but the difference between one motor and two motors is the most pronounced.

So, is the extra money for a dual-motor version worth it in reality? I would argue that it is not. But not because it is bad. The key reason is that electric motors are so good that even just one of them is fully enough in almost all cases.

There are two situations where you'll notice a difference: very rapid acceleration (at either very low or very high speeds) and traction in very slippery situations, like driving uphill on a snow- or ice-covered road.

When accelerating, the dual-motor configurations ... just teleport to the new location and new speed. The feeling is just crazy. If you stab the accelerator pedal in such a car, you better have very good reflexes. I have out-accelerated ICE cars worth over a million euros, leaving their owners literally speechless. The single-motor version, in comparison, is "just" a very fast car. Like a Porsche. The acceleration is still very fast, but the car is not just "teleporting" and is accelerating over a time that a human brain can still comprehend. The only downside (and that is only relevant in Germany) is the top speed -- the single-motor variants are typically locked to 180-190 km/h instead of the 225-230 km/h that dual-motor variants are allowed to reach. And to be fair, BMW cars -- especially the i5 M60 -- feel extremely stable and safe at speeds over 200 km/h, so for some people in Germany, that could be a real argument for an M-Performance car. It also does not consume all that much more energy at high speeds, unlike its gasoline siblings.

When driving in a slippery situation (I have driven both in snow, ice, and mud), the dual-drive version is just magical in its ability to just keep driving without spinning the wheels in any conditions. But on all normal roads (including climbing an uphill road during a blizzard), I was also able to slowly drive forward with the single-motor i5 as well. The much better traction control that an electric motor allows makes a BEV RWD car drive almost as easily in bad road conditions as an AWD ICE car can. The key is that a BEV has maximum torque available from zero RPM. That means you do not need to press the accelerator pedal more to have enough torque to move the car up the hill and thus risk the wheels spinning out. A BEV can roll the car uphill even at the lowest speed.

Cargo area

One big difference between the i4 and i5 is the rear cargo area -- the i4 has a huge hatch opening and the whole rear window goes up. You can basically put anything in there. The i4 also does not have a fixed crossbar behind the rear seats, so with the seats down you get a fully open cargo area from the floor to the roof. Very practical. The i5, in contrast, is more conservative -- it is a classical sedan with a narrower opening. The rear window stays fixed and there is also a fixed, non-removable crossbrace and shelf behind the rear seats. On the other hand, the length of the cargo area is much deeper; you can basically fit a couple of extra crates of beer in the i5. Also, the underfloor compartment in the rear of the i5 is more usable than in the i4, as in the i4 that is also where they put the subwoofer of the high-end sound system. In the i5, it is hidden deeper in the car and does not take up any cargo space.

Seats

The rear seat of the i5 is also far more usable for adult passengers. One can go as far as to say that the rear seats in the i5 are as comfortable as normal front seats in other cars. But the i5 goes a step further as there is an option for comfort seats in the front, and I enjoyed that a lot. Three years ago, I had a very painful (but necessary) surgery on the bones in my chest. This made me much more sensitive to back, neck, and shoulder pain. In all other cars (including the i4), I was getting back pain after 3-4 hours of driving and needed to spend all the charging stops exercising during day-long drives. After we got the i5 with comfort seats, one of the first things we did was a long weekend trip from Ulm to... Barcelona. That was basically three full days of driving across a four-day period. I felt as fresh and pain-free when we came back from that trip as when we left. Sadly, there was no massage seat option for the i5 -- a rare miss, I think. But the ventilated seats were quite nice during the hot summer months.

Driving assistance

The i5 has the same kind of driving assistance as the i4 had -- rock-solid adaptive cruise control and rock-solid lane keeping. It features great adjustment of driving speed in anticipation of conditions, such as slowing down before tight corners, before intersections where navigation expects a turn to happen, and before roundabouts. It also slows down in advance of known upcoming lower speed limit signs so that the car crosses the speed limit sign line already driving the new speed limit, as required by most traffic laws. This is in contrast to other systems that only recognize the new speed limit and only start to slow down a couple of seconds after passing the speed limit sign.

Naturally, something extra was also developed in the couple of years between those models, like the Highway Assistant that allows fully legal hands-free (but still eyes-on-the-road) operation up to the legally limited 135 km/h. I initially dismissed this as a pure gimmick, but after using it on a few longer drives, I can tell that the difference in stress in my arms and shoulders is profound. Just being able to keep both arms down in my lap or on handrests does wonders for having a more relaxed driving experience. The system monitors the driver's gaze even more strictly in this mode -- you have to watch the road. As a benefit of that, the system also can detect when you are checking the side mirrors, so when the system detects that it would be beneficial to change the lane, you can confirm this lane change by simply checking the right or left side mirror. Or you can ask the car to do the lane change by activating the turn indicators. If anything, that is a great way to get BMW drivers to use those things :D

Parking assistance is also improved, including a function to park the car "remotely" by using the phone as a dead-man's switch for a pre-selected parking maneuver. This worked reliably, including underground, with zero signal. I also got a lot of use out of the pre-programmed parking routes function -- you can show the car a parking maneuver once and it will then be able to repeat it at the same location, even if the situation is a bit different every time. This works for routes up to ~40 meters long. So I can basically make a route from the entrance of the parking garage of my apartment complex all the way to my parking spot. I only need to drive it well once; after that, I can just let the car take over at any point in the path. This also works if the parking garage gets no cellular (or WiFi) signal and no GPS either. If there is a new, unavoidable obstacle in the pre-programmed way, the car will see it with ultrasonics and will just stop safely. And you can slow down or stop the automatic motion at any moment with the brake pedal.

BMW retains the concept of cooperative driver assistance here -- the car is not taking over the full driving from you; it is cooperating with you. So, for example, while driving on the highway with full cruise control and lane keeping enabled, I can still lightly tug on the steering wheel left to make the car drive in the left side of the lane instead of the middle to avoid a pothole or give a bit more distance to someone stopped on the shoulder.

Software

The BMW i5 is built on the MGU22 platform, which is an evolution of the MGU21 (and MGU18) platform that was in my previous BMW i4 car and that I described in my Debconf talk back in Montreal.

It is still based on a custom Linux build compiled with Yocto with GENIVI protocols like DLT and SomeIP connecting all the components. The UI is still running on Wayland. The screens are a bit larger than before, the resolution is higher, the responsiveness is better, and there are more apps doing different things. But as a user, it would be hard for me to find noticeable differences between these specific product generations. Part of it is on purpose -- to help users retain their user experience as they upgrade from one BMW to the next one.

Automatic migration of preferences and saved navigation locations from car to car via the BMW account login also helps that process. It can be amazing for people in corporate fleets or large (and well-off) families where one can expect to periodically drive different BMW cars (of the same model) and enjoy the seamless transfer of all their preferences (including things like seat positions) to the new car. This could have been more useful if a cross-brand standard for user in-car preferences ever existed. Maybe that will happen in the future, when cars become more of a shared commodity.

One big development that happened while I was using the i5, which also applies to all other BMW cars, is the rapid and impressive development of the MyBMW app. The team behind that has been doing great work over the past couple of years and delivered a lot of features and enhancements to user experience. For example, the automatic suggestion of destinations based on the location of events in your calendar happens (if you opt-in to this) via the MyBMW app.

Another thing that often stays unremarked is that BMW is managing the largest fleet of cars with automatic, remote software update capability in the world. ALL of the BMW models have this capability (not just BEVs) and BMW has been manufacturing ~2.5 million such cars every year since about 2018. The total remote software update capable fleet is more than twice as large as that of Tesla, for example. And it is also more complex as it has to update software on nearly 60 different models across nearly all countries in the world. This system does not just update the infotainment ECU in the car -- it updates ALL the ECUs in the car (if needed). And that is also a huge and very challenging task. The fact that it just works, with failures measured in single digits per millions of updates (mostly related to damaged hardware), is remarkable, but often stays unremarked.

Charging

Basically, everything that I said about the i4 before also applies to the i5. The charging is a bit better -- the charging curve is flatter, so it matters even less to "manage" the charging perfectly. The consumption is a bit better -- likely due to the single-motor configuration. High-speed consumption is astonishingly good -- I was routinely cruising long distances at 190 km/h and never had a feeling that this high speed impacted my range much.

In the end, I don't even talk about charging speeds or consumption anymore. Instead, in real life, I think about charging and consumption in terms of time, and then it sounds like this:

• In Germany (speeds around 180 km/h): from full charge, drive ~3 hours, charge for 20 minutes, drive another ~2 hours
• On highways outside Germany (~140 km/h): from full charge, drive ~4 hours, charge for 20 minutes, drive another ~3 hours
• On regional roads (90–100 km/h): from full charge, drive ~6 hours, charge for 15 minutes, and drive another ~4 hours

All I do normally is:

• On any longer drive, just enter all destinations for today into the BMW navigation system, then it will automatically plan for the best charging location. If at the end of the day I will not get to a place with an overnight charger and I will need to keep driving further, then I also add tomorrow's first stop into the navigation.
• In everyday usage, I simply prefer to park in locations that have a charger (any charger).

If I go downtown to a cinema, there is a parking garage there that has 11 kW chargers that will pump 20–30% of battery capacity into the car while I am watching the movie. If I go shopping, I choose a shop that has a 50–150 kW charger nearby -- then in the half an hour that I usually spend in the shop, the car will usually be up to 80–90% capacity. Even going to a shopping mall that has 11 kW chargers is worth it. Even if I just charge up 10% of the battery while in the shop, I still get home with more battery than I left with. This way, even without a charger at home, I can easily maintain or increase the charge level of my car's battery without having to explicitly go somewhere just to charge.

Charging works best when it happens incidentally -- where and when your car is parked anyway.

I hope to get a charging station at home this year; then I will only be charging outside of the home when I drive more than ~4 hours on the Autobahn on the same day. Just driving from Ulm to Munich and back would then not be enough to make me have to charge outside of home.

There is a negative nuance in charging an electric car in Europe, and Germany specifically. The costs have increased a lot and flexibility has decreased a lot in the past few years. The cost of electricity shot up a lot after the attack on Ukraine started, and the cost of public charging soon followed. And while the retail electricity cost has decreased again, the cost of charging is still high. Especially for corporate roaming cards that company car users are using. It is not unusual to see costs of 1 €/kWh and blocking fees of 17 €/hour after as little as 30 minutes from the start of charging. The EU needs to pay more attention to this as such costs are a real blocker for electromobility. Luckily, it is still quite easy to find chargers costing as little as 0.35–0.49 €/kWh and having much more reasonable blocking fees that only start after 1 hour on fast chargers and 4 hours (during the day) on slow chargers.

What's next?

So my BMW i5 is back in the dealership, so what is next for me? Unusually, I will be spending the next few months without a car and reliant on the great public transport and the amazing Deutschlandticket that, for a low price (just over 40 €/month after employer discount), gives me access to all regional and city public transport across the entire Germany. It does not include long-distance high-speed trains (like ICE), but all local transport is there. And that will also save me some money on the taxable benefit side as well from not having a company car.

I would not do that voluntarily, so why? Well, I am waiting for the new car -- the BMW iX3 Neue Klasse.

This is going to be the first car on the Neue Klasse platform, also known as the Service Pack 25 architecture. The previous cars were on the SP21 architecture. This is going to be a BIG deal. In this generation, everything is new and redesigned from scratch. The whole concept of what a car is and what a BMW is was recreated from a blank slate. And all of that was done with BEV design as basically the only design choice. The NK design can include range-extending gasoline or hydrogen engines, but the main driving force in NK always is the big battery and the strong electrical motor. Or two. Or three. Or four.

It is a whole new world, let me tell you. This car, compared to all previous BMW cars, feels like a generation was skipped. It is two generations ahead of everything BMW has done before. In electromobility, in what a car is, in traction and stability, in software, in user experience. I've been working on this for, basically, the last 5 years. A lot has been invested into this.

And it is not just one car. All BMW cars in the future will transition to Neue Klasse designs. This is a clean, architectural break. This is what "a BMW" will mean for the next decades. It is really hard to overstate the importance of this particular car model. And that is why I am happy to wait a few months without a car for it. I am hoping for delivery in April, but demand is so high that I am not certain when I'll get it, even if I ordered the car the very first day that a configurator was available. The whole planned production capacity for Europe for 2026 is supposedly already sold out.

What can I tell about the software there? Not much more than is already published. It is still a Linux-based operating system, but now you will notice that it is also based on Android, specifically on the AOSP software stack. It still remains a 100% in-house developed OS and software stack with full control over all integrations. As shown just last week at CES, there is a local LLM for an in-car voice assistant and an integration to off-car Alexa+ expansion to cover wider services, like rich location search and hands-free booking of a table at a restaurant, for example. Naturally, each part of these services needs to meet strict privacy requirements, and the user needs to be able to agree or disagree to use them. And all of that needs to be tested.

I hope that I can talk a bit more about this in one of the future Debconfs again.

Questions? Feedback? Just ask here or here.

Snapshot mirroring in Debian (and Ubuntu)

The use of snapshots has been routine in both Debian and Ubuntu for several years now—or more than 15 years for Debian, to be precise. Snapshots have become not only very reliable, but also an increasingly important part of the Debian package archive.

This week, I encountered a problem at work that could be perfectly solved by correctly using the Snapshot service. However, while trying to figure it out, I ran into some shortcomings in the documentation. Until the docs are updated, I am publishing this blog post to make this information easier to find.

Problem 1: Ensure fully reproducible creation of Docker containers with the exact same packages installed, even years after the original images were generated.

Solution 1: Pin everything! Use a pinned source image in the FROM statement, such as debian:trixie-20250721-slim, and also pin the APT package sources to the "same" date - "20250722".

Hint: The APT packages need to be newer than the Docker image base. If the APT packages are a bit newer, that's not a problem, as APT can upgrade packages without issues. However, if your Docker image has a newer package than your APT package sources, you will have a big problem. For example, if you have "libbearssl0" version 0.6-2 installed in the Docker image, but your package sources only have the older version 0.6-1, you will fail when trying to install the "libbearssl-dev" package. This is because you only have version 0.6-1 of the "-dev" package available, which hard-depends on exactly version 0.6-1 of "libbearssl0", and APT will refuse to downgrade an already installed package to satisfy that dependency.

Problem 2: You are using a lot of images in a lot of executions and building tens of thousands of images per day. It would be a bad idea to put all this load on public Debian servers. Using local sources is also faster and adds extra security.

Solution 2: Use local (transparently caching) mirrors for both the Docker Hub repository and the APT package source.

At this point, I ran into another issue—I could not easily figure out how to specify a local mirror for the snapshot part of the archive service.

First of all, snapshot support in both Ubuntu and Debian accepts both syntaxes described in the Debian and Ubuntu documentation above. The documentation on both sites presents different approaches and syntax examples, but both work.

The best approach nowadays is to use the "deb822" sources syntax. Remove /etc/apt/sources.list (if it still exists), delete all contents of the /etc/apt/sources.list.d directory, and instead create this file at /etc/apt/sources.list.d/debian.sources:

Types: deb
URIs: https://common.mirror-proxy.local/ftp.debian.org/debian/
Suites: trixie
Components: main non-free-firmware non-free contrib
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
Snapshot: 20250722

Hint: This assumes you have a mirror service running at common.mirror-proxy.local that proxies requests (with caching) to whitelisted domains, based on the name of the first folder in the path.

If you now run sudo apt update --print-uris, you will see that your configuration accesses your mirror, but does not actually use the snapshot.

Next, add the following to /etc/apt/apt.conf.d/80snapshots:

APT::Snapshot "20250722";

That should work, right? Let's try sudo apt update --print-uris again. I've got good news and bad news! The good news is that we are now actually using the snapshot we specified (twice). The bad news is that we are completely ignoring the mirror and going directly to snapshots.debian.org instead.

Finding the right information was a bit of a challenge, but after a few tries, this worked: to specify a custom local mirror of the Debian (or Ubuntu) snapshot service, simply add the following line to the same file, /etc/apt/apt.conf.d/80snapshots:

Acquire::Snapshots::URI::Override::Origin::debian "https://common.mirror-proxy.local/snapshot.debian.org/archive/debian/@SNAPSHOTID@/";

Now, if you check again with sudo apt update --print-uris, you will see that the requests go to your mirror and include the specified snapshot identifier. Success!

Now you can install any packages you want, and everything will be completely local and fully reproducible, even years later!

Debconf 25 photos

Debconf 25 came to the end in Brest, France this year a couple weeks ago.

This has been a very different and unusually interesting Debconf. For me it was for two, related reasons: for one the conference was close enough in Western Europe that I could simply drive there with a car (which reminds me that I should make a blog post about the BMW i5, before I am done with it at the end of this year) and for the other - the conference is close enough to Western Europe that many other Debian developers could join this year who have not been seen at the event for many years. Being able to arrive early, decompress and spend extra time looking around the place made the event itself even more enjoyable than usual.

The French cuisine, especially in its Breton expression, has been a very welcome treat. Even if there were some rough patches with the food selection, amount, or waiting, it was still a great experience.

I specifically want to say a big thank you to the organisers for everything, but very explicitly for planning all the talk/BOF rooms in the same building and almost on the same floor. It saved me a lot of footwork, but also for other participants the short walks between the talks made it possible to always have a few minutes to talk to people or grab a croissant before running to the next talk.

IMHO we should come back to a tradition of organising Debconf in Europe every 2-3 years. This maximises one of the main goals of Debconf - bringing as many Debian Developers as possible together in one physical location. This works best when the location is actually close to large concentrations of existing developers. In other years, the other goal of Debconf can then take priority - recruiting new developers in new locations. However, these goals could both be achieved at the same time - there are plenty of locations in Europe and even in Western Europe that still have good potential for attracting new developers. Especially if we focus on organising the event on the campuses of some larger technically-oriented universities.

This year was also very productive for me—a lot of conversations with various people about all kinds of topics, especially technical packaging questions. It has been a long time since the very basic foundations of Debian packaging work have been so fundamentally refactored and modernized as in the past year. Tag2upload has become a catalyst for git-based packaging and for automated workflows via Salsa, and all of that feeds back into focusing on a few best-supported packaging workflows. There is still a bit of a documentation gap of a new contributor getting to these modern packaging workflows from the point where the New Maintainers Guide stops.

In any case, next year Debconf will be happening in Santa Fe, Argentina. And the year after that it is all still open and in a close competition between Japan, Spain, Portugal, Brazil and .. El Salvador? Personally, I would love to travel to Japan (again), but Spain or Portugal would also be great locations to meet more European developers again.

As for Santa Fe ... it is quite likely that I will not be able to make it there next year, for (planned) health reasons. I guess I should also write a new blog post about what it means to be a Debconf Photographer, so that someone else could do this as well, and also reduce the "bus factor" along the way.

But before that - here is the main group photo from this year:

You can also see it on:

• Google Photos

You can also enjoy the rest of the photos:

• on git-lfs
• on Google Photos

Additionally, check out photos from other people on GIT LFS and consider adding your own photos there as well.

Other places I have updated with up-to-date information are these wiki pages:

• All Debconf Group Photos
• All photos from Debconf25

If you took part in the playing cards event, then check your photo in this folder and link to your favourite from your line in the playing card wiki

Debconf 24 is coming to a close in Busan, South Korea this year.

I thought that last year in India was hot. This year somehow managed to beat that. With 35C and high humidity the 55 km that I managed to walk between the two conference buildings have really put the pressure on. Thankfully the air conditioning in the talk rooms has been great and fresh water has been plentiful. And the korean food has been excellent and very energetic.

Today I will share with you the main group photo:

You can also see it in:

• on Google Photos
• on git-lfs

The rest of my photos from the event will be published next week. That will give me a bit more time to process them correctly and also give all of you a chance to see these pictures with fresh eyes and stir up new memories from the event.

At the end of the last part of this, we got a Home Assistant OS installation that contains in itself a Firefly III instance and that contains all the current financial information. They are communicating and calculating predictions for me.

The only part that I was not 100% happy with was accounting of cash transactions. You see payments in cash are mostly made away from computer and sometimes even in areas without a mobile Internet connection. And all Firefly III apps that I tried failed at the task of creating a new transaction when offline. Even the one recommended Telegram bot from Firefly III page used a dialog-based approach for creating even a simplest transaction. Issue asking for a one-shot transaction creation option stands as unresolved.

Theoretically it would have been best if I could simply contribute that feature to that particular Telegram bot ... but it's written in Javascript. By mapping the API onto tasks somehow. After about 4 hours I still could not figure out where in the code anything actually happens. It all looked like just sugar or spagetty. Connectors on connectors on mappers.

So I did the real open-source thing and just wrote my own tool. firefly3_telegram_oneshot is a maximally simple Telegram bot based on python-telegram-bot library.

So what does it do? The primary usage for me is to simply send a message to the bot at any time with text like "23.2 coffee and cake" and when the message eventually reaches the bot, it then should create a new transaction from my "cash" account to "Unknown" account in amount of 23.20€ and description "coffee and cake".

That is the key. Everything else in the bot is comfort.

For example "/undo" command deletes the last transaction in cash account (presumably added by error) and "/last" shows which transaction the "/undo" would delete.

And to help with expense categorisation one can also do a message like "6.1 beer, dest=Edeka, cat=alcohol" that would search for a destination account that would fuzzy match to "Edeka" (a supermarket in Germany) and add the transation to the category fuzzy matched to "alcohol", like "Shopping - alcoholic drinks".

And to make that fuzzy matching more reliable I also added "/cat something" and "/dest something" commands that would show which category or destination account would be matched with a given string.

All of that in around 250 lines of Python code and executed by a 17 line Dockerfile (via the Portainer on my Home Assistant OS). One remaining function that could be nice is creating a category or destination account on request (for example when the first character of the supplied string is "+").

I am really plesantly surprised about how much can be done with how little code using the above Python library. And you never need to have any open incoming ports anywhere to runs such bots, so the attack surface for such bot-based service is much tighter.

All in all the system works and works well. The only exception is that for my particual bank there is still no automatic way of extracting data about credit card transactions. For those I still have to manually log into the Internet bank, export a CSV file and then feed that into the Firefly III importer. Annoying. And I am not really motivated to try to hack my bank :D

Has this been useful to any of you? Any ideas to expand or improve what I have? Just find me as "aigarius" on any social media and speak up :)

At the end of the last part of this, we got a Home Assistant OS installation that contains in itself a Firefly III instance and that contains all the current financial information. Now I will try to connect the two.

While it could be nice to create a fully-featured integration for Firefly III to Home Assistant to communicate all interesting values and events, I have an interest on programming a more advanced data point calculation for my budget needs, so a less generic, but more flexible approch is a better one for me. So I was quite interested when among the addons in the Home Assistant Addon Store I saw AppDaemon - a way to simply integrate arbitrary Python processing with Home Assistant. Let's see if that can do what I want.

For start, after reading the tutorial , I wanted to create a simple script that would use Firefly III REST API to read the current balance of my main account and then send that to Home Assistant as a sensor value, which then can be displayed on a dashboard.

As a quick try I modified the provided hello_world.py that is included in the default AppDaemon installation:

import requests
from datetime import datetime
import appdaemon.plugins.hass.hassapi as hass
app_token = "<FIREFLY_PERSONAL_ACCESS_TOKEN>"
firefly_url = "<FIREFLY_URL>"

class HelloWorld(hass.Hass):
    def initialize(self):
        self.run_every(self.set_asset, "now", 60 * 60)

    def set_asset(self, kwargs):
        ent = self.get_entity("sensor.firefly3_asset_sparkasse_main")
        if not ent.exists():
            ent.add(
                state=0.0,
                attributes={
                    "native_value": 0.0,
                    "native_unit_of_measurement": "EUR",
                    "state_class": "measurement",
                    "device_class": "monetary",
                    "current_balance_date": datetime.now(),
                })

        r = requests.get(
            firefly_url + "/api/v1/accounts?type=asset",
            headers={
                "Authorization": "Bearer " + app_token,
                "Accept": "application/vnd.api+json",
                "Content-Type": "application/json",
        })
        data = r.json()
        for account in data["data"]:
            if not "attributes" in account or "name" not in account["attributes"]:
                continue
            if account["attributes"]["name"] != "Sparkasse giro":
                continue
            self.log("Account :" + str(account["attributes"]))
            ent.set_state(
                state=account["attributes"]["current_balance"],
                attributes={
                    "native_value": account["attributes"]["current_balance"],
                    "current_balance_date": datetime.fromisoformat(account["attributes"]["current_balance_date"]),
                })
            self.log("Entity updated")

It uses a URL and personal access token to access Firefly III API, gets the asset accounts information, then extracts info about current balance and balance date of my main account and then creates and/or updates a "sensor" value into Home Assistant. This sensor is with metadata marked as a monetary value and as a measurement. This makes Home Assistant track this value in the database as a graphable changing value.

I modified the file using the File Editor addon to edit the /config/appdaemon/apps/hello.py file. Each time the file is saved it is reloaded and logs can be seen in the AppDaemon Logs section - main_log for logging messages or error_log if there is a crash. Useful to know that requests library is included, but it hard to see in the docks what else is included or if there is an easy way to install extra Python packages.

This is already a very nice basis for custom value insertion into Home Assistant - whatever you can with a Python script extract or calculate, you can also inject into Home Assistant. With even this simple approach you can monitor balances, budgets, piggy-banks, bill payment status and even sum of transactions in particular catories in a particular time window. Especially interesting data can be found in the insight section of the Firefly III API.

The script above uses a trigger like self.run_every(self.set_asset, "now", 60 * 60) to simply run once per hour. The data in Firefly will not be updated too often anyway, at least not until we figure out how to make bank connection run automatically without user interaction and not screw up already existing transactions along the way. In theory a webhook API of the Firefly III could be used to trigger the data update instantly when any transaction is created or updated. Possibly even using Home Assistant webhook integration. Hmmm. Maybe.

Who am I kiddind? I am going to make that work, for sure! :D But first - how about figuring out the future?

So what I want to do? In short, I want to predict what will be the balance on my main account just before the next months salary comes in. To do this I would:

• take the current balance of the main account
• if this months salary is not paid out yet, then add that into the balance
• deduct all still unpaid bills that are due between now and the target date
• if the credit card account has not yet been reset to the main account, deduct current amount on the cards
• if credit card account has been reset, but not from main account deducted yet, deduct the reset amount

To do that I need to use the Firefly API to read: current account info, status of all bills including next due date and amount, transfer transactions between credit cards and main account and something that would store the expected salary date and amount. Ideally I'd use a recurring transaction or a income bill for this, but Firefly is not really cooperating with that. The easiest would be just to hardcode that in the script itself.

And this is what I have come up with so far.

To make the development process easier, I separated put the params for the API key and salary info and app params for the month to predict for, and predict both this and next months balances at the same time. I edited the script locally with Neovim and also ran it locally with a few mocks, uploading to Home Assistant via the SSH addon when the local executions looked good.

So what's next? Well, need to somewhat automate the sync with the bank (if at all possible). And for sure take a regular database and config backup :D

So now that I have something that looks very much like a budgeting setup going, I am going to .. delete it! Why? Well, at the end of the last part of this, the Firefly III instance was running on a tiny Debian server in a Docker container right next to another Docker container that is running the main user of this server - a Home Assistant instance that has been managing my home for several years already. So why change that?

See, there is one bit of knowledge that is very crucial to your Home Assistant experience, which is not really emphasised enough in the Home Assistant documentation. In fact back when I was getting into the Home Assistant both the main documentation and basically all the guides around were just coming off the hype of Docker disrupting everything and that is a big reason why everyone suggested to install and use Home Assistant as a Docker container on top of any kind of stable OS. In fact I used to run it for years on my TerraMaster NAS, just so that I don't have a separate home server running 24/7 at home and just have everything inside the very compact NAS case.

So here is the thing you NEED to know - Home Assistant Container is DEMO version of Home Assistant! If you want to have a full Home Assistant experience and use the knowledge of the huge community around the HA space, you have to use the Home Assistant OS. Ideally on dedicated hardware. Ideally on HA Green box, but any tiny PC would also work great. Raspberry Pi 4+ is common, but quite weak as the network size grows and especially the SD card for storage gets old very fast. Get a real small x86 PC with at least 4Gb RAM and a NVME SSD (eMMC is fine too). You want to have an Ethernet port and a few free USB ports. I would also suggest immediately getting HA SkyConnect adapter that can do Zigbee networking and will do Matter soon (tm). I am making do with a SonOff Zigbee gateway, but it is quite hacky to get working and your whole Zigbee communication breaks down if the WiFi goes down - suboptimal.

So I took a backup of the Home Assistant instance using it's build-in tools. I took an export of my fully configured Firefly III instance and proceeded to wipe the drive of the NUC. That was not a smart idea. :D

On the Home Assitant side I was really frustrated by the documentation that was really focused on users that are (likely) using Windows and are using an SD card in something like Raspberry Pi to get Home Assistant OS running. It recommended downloading Etcher to write the image to the boot medium. That is a really weird piece of software that managed to actually crash consistently when I was trying to run it from Debian Live or Ubuntu Live on my NUC. It took me way too long to give up and try something much simpler - dd.

xzcat haos_generic-x86-64-11.0.img.xz | dd of=/dev/mmcblk0 bs=1M

That just worked, prefectly and really fast. If you want to use a GUI in a live environment, then just using the gnome-disk-utility ("Disks" in Gnome menu) and using the "Restore Disk Image ..." on a partition would work just as well. It even supports decompressing the XZ images directly while writing.

But that image is small, will it not have a ton of unused disk space behind the fixed install partition? Yes, it will ... until first boot. The HA OS takes over the empty space after its install partition on the first boot-up and just grows its main partition to take up all the remaining space. Smart. After first boot is completed, the first boot wizard can be accessed via your web browser and one of the prominent buttons there is restoring from backup. So you just give it the backup file and wait. Sadly the restore does not actually give any kind of progress, so your only way to figure out when it is done is opening the same web adress in another browser tab and refresh periodically - after restoring from backup it just boots into the same config at it had before - all the settings, all the devices, all the history is preserved. Even authentification tokens are preserved so if yu had a Home Assitant Mobile installed on your phone (both for remote access and to send location info and phone state, like charging, to HA to trigger automations) then it will just suddenly start working again without further actions needed from your side. That is an almost perfect backup/restore experience.

The first thing you get for using the OS version of HA is easy automatic update that also automatically takes a backup before upgrade, so if anything breaks you can roll back with one click. There is also a command-line tool that allows to upgrade, but also downgrade ha-core and other modules. I had to use it today as HA version 23.10.4 actually broke support for the Sonoff bridge that I am using to control Zigbee devices, which are like 90% of all smart devices in my home. Really helpful stuff, but not a must have.

What is a must have and that you can (really) only get with Home Assistant Operating System are Addons. Some addons are just normal servers you can run alongside HA on the same HA OS server, like MariaDB or Plex or a file server. That is not the most important bit, but even there the software comes pre-configured to use in a home server configuration and has a very simple config UI to pre-configure key settings, like users, passwords and database accesses for MariaDB - you can litereally in a few clicks and few strings make serveral users each with its own access to its own database. Couple more clicks and the DB is running and will be kept restarted in case of failures.

But the real gems in the Home Assistant Addon Store are modules that extend Home Assitant core functionality in way that would be really hard or near impossible to configure in Home Assitant Container manually, especially because no documentation has ever existed for such manual config - everyone just tells you to install the addon from HA Addon store or from HACS. Or you can read the addon metadata in various repos and figure out what containers it actually runs with what settings and configs and what hooks it puts into the HA Core to make them cooperate. And then do it all over again when a new version breaks everything 6 months later when you have already forgotten everything. In the Addons that show up immediately after installation are addons to install the new Matter server, a MariaDB and MQTT server (that other addons can use for data storage and message exchange), Z-Wave support and ESPHome integration and very handy File manager that includes editors to edit Home Assitant configs directly in brower and SSH/Terminal addon that boht allows SSH connection and also a web based terminal that gives access to the OS itself and also to a comand line interface, for example, to do package downgrades if needed or see detailed logs. And also there is where you can get the features that are the focus this year for HA developers - voice enablers.

However that is only a beginning. Like in Debian you can add additional repositories to expand your list of available addons. Unlike Debian most of the amazing software that is available for Home Assistant is outside the main, official addon store. For now I have added the most popular addon repository - HACS (Home Assistant Community Store) and repository maintained by Alexbelgium. The first includes things like NodeRED (a workflow based automation programming UI), Tailscale/Wirescale for VPN servers, motionEye for CCTV control, Plex for home streaming. HACS also includes a lot of HA UI enhacement modules, like themes, custom UI control panels like Mushroom or mini-graph-card and integrations that provide more advanced functions, but also require more knowledge to use, like Local Tuya - that is harder to set up, but allows fully local control of (normally) cloud-based devices. And it has AppDaemon - basically a Python based automation framework where you put in Python scrips that get run in a special environment where they get fed events from Home Assistant and can trigger back events that can control everything HA can and also do anything Python can do. This I will need to explore later.

And the repository by Alex includes the thing that is actually the focus of this blog post (I know :D) - Firefly III addon and Firefly Importer addon that you can then add to your Home Assistant OS with a few clicks. It also has all kinds of addons for NAS management, photo/video server, book servers and Portainer that lets us setup and run any Docker container inside the HA OS structure. HA OS will detect this and warn you about unsupported processes running on your HA OS instance (nice security feature!), but you can just dismiss that. This will be very helpful very soon.

This whole environment of OS and containers and apps really made me think - what was missing in Debian that made the talented developers behind all of that to spend the immense time and effor to setup a completely new OS and app infrastructure and develop a completel paraller developer community for Home Assistant apps, interfaces and configurations. Is there anything that can still be done to make HA community and the general open source and Debian community closer together? HA devs are not doing anything wrong: they are using the best open source can provide, they bring it to people whould could not install and use it otherwise, they are contributing fixes and improvements as well. But there must be some way to do this better, together.

So I installed MariaDB, create a user and database for Firefly. I installed Firefly III and configured it to use the MariaDB with the web config UI. When I went into the Firefly III web UI I was confronted with the normal wizard to setup a new instance. And no reference to any backup restore. Hmm, ok. Maybe that goes via the Importer? So I make an access token again, configured the Importer to use that, configured the Nordlinger bank connection settings. Then I tried to import the export that I downloaded from Firefly III before. The importer did not auto-recognose the format. Turns out it is just a list of transactions ... It can only be barely useful if you first manually create all the asset accounts with the same names as before and even then you'll again have to deal with resolving the problem of transfers showing up twice. And all of your categories (that have not been used yet) are gone, your automation rules and bills are gone, your budgets and piggy banks are gone. Boooo. It will be easier for me to recreate my account data from bank exports again than to resolve data in that transaction export.

Turns out that Firefly III documenation explicitly recommends making a mysqldump of your own and not rely on anything in the app itself for backup purposes. Kind of sad this was not mentioned in the export page that sure looked a lot like a backup :D

After doing all that work all over again I needed to make something new not to feel like I wasted days of work for no real gain. So I started solving a problem I had for a while already - how do I add cash transations to the system when I am out of the house with just my phone in the hand? So far my workaround has been just sending myself messages in WhatsApp with the amount and description of any cash expenses. Two solutions are possible: app and bot.

There are actually multiple Android-based phone apps that work with Firefly III API to do full financial management from the phone. However, after trying it out, that is not what I will be using most of the time. First of all this requires your Firefly III instance to be accessible from the Internet. Either via direct API access using some port forwarding and secured with HTTPS and good access tokens, or via a VPN server redirect that is installed on both HA and your phone. Tailscale was really easy to get working. But the power has its drawbacks - adding a new cash transaction requires opening the app, choosing new transaction view, entering descriptio, amount, choosing "Cash" as source account and optionally choosing destination expense account, choosing category and budget and then submitting the form to the server. Sadly none of that really works if you have no Internet or bad Internet at the place where you are using cash. And it's just too many steps. Annoying.

An easier alternative is setting up a Telegram bot - it is running in a custom Docker container right next to your Firefly (via Portainer) and you talk to it via a custom Telegram chat channel that you create very easily and quickly. And then you can just tell it "Coffee 5" and it will create a transaction from the (default) cash account in 5€ amount with description "Coffee". This part also works if you are offline at the moment - the bot will receive the message once you get back online. You can use Telegram bot menu system to edit the transaction to add categories or expense accounts, but this part only work if you are online. And the Firefly instance does not have to be online at all. Really nifty.

So next week I will need to write up all the regular payments as bills in Firefly (again) and then I can start writing a Python script to predict my (financial) future!

A week ago I started to migrate my financial planning from a closed source system to a new system based on an open source, self-hosted solution. Main candidate is Firefly III - a relatively simple financial planner with a rather rich feature set and a solid user base and developer support.

Starting it up with a Docker-Compose file was quite easy, following the official documentation. The same Compose file also managed the MySQL database, the importer app and a cron container for regular imports. The separate importer app allows both imports from CSV files and also from external bank account connection services. For whatever weird reason both of those services support exporting data from my regular bank accounts, but not from my credit cards for exactly the same bank. So for those cards I would need to periodically download CVS transaction exports and feed them into the importer.

The combination of the cron container and the importer app allows for both of these functions. To do this you first do the import via the Web UI first and configure all the mappings - configure file and date formats, map account names in the bank output to account names that you added in Firefly, configure what otehr columns mean and what is the mapping of other values, like expense and income accounts. At the end of the process you can download a json file representing all the settings of the import you just did. Putting such a json config file in the input folder of the cron container and telling it to do an import (weekly, for example) would do such import periodically. Putting a credit card CSV file along with a config file for credit card import would also auto-import that.

So far so good. However, when I tried importing exports from MoneyWiz or even when I tried re-creating account history directly from the bank data, I hit a very annoying problem. Transfers. Thing is detecting transfers between the real (asset) accounts that you are managing is really essential. For one those are not real expenses and incomes, so failing to mark them as transfers would show weird income/expense numbers. But if you do detect them as transfers and correctly map the destination account, but fail to match the transations between another you get a double-transaction. This becomes really hard when transactions happen on different dates and have different descriptions. So you get both a +1000€ transaction "Credit card reset" on 14.09.2023 in the credit card account and a -1000$ transaction "Repayment of own credit card" on 24.09.2023 in the main account of the bank. Matching them to recognise that it is just one transfer and not two is ... non-trivial. The best solution I could come up with so far is to always map the "Opposing account name" for such transfers to a virtual "Transfers" asset account. That has the benefit of being actually able to correctly represent transfers that take several days to move between accounts and showing you how much money is still in transit at any point in time.

So after I figured this out, finally the account balances started matching up with reality.

Setting up spending categories required another change - the author of the Firefly III does not like complexity so it does not support nesting categories. Categories can only be in a single, flat list. It is suggested that if you do need to track multiple categories and also their combination, then category groups might help you. I will survive for now by simplifying the categories that I do actually use. Might actually make the reports more usable.

A new feature for me will be the ability to use automation rules to assign transactions to categories based on their contents, like expense accounts of keywords in descriptions.

Setting up regular bills (with matching rules to assign incoming transactions to those bills) is another feature that is very important to me. The feature itself works just fine in Firefly III, but it has two restrictions that the author of the software does not actually want to be changed. For one it can not be used to track prodictable incomes (like salary), presumably because it is only there for bills (subscriptions in v3 UI). And for other, there is nothing in the base software that actually uses the data from bills to look forward. The author does not like to try to predict the future. Which for me is basically the one of two reasons to use this kind of software at all. I want to know - given all manually entered regular and 100% predictable expenses and incomes, what will be the state of my accounts at particular date? It seems that to get at that I will need to write my own oracle script using the rich Firefly III API.

The last question I had for this week was - how will I enter a new cash puchace of potatoes on a farmers market into the system that sits in a private server in my home network? Turns out there are actually multiple Android apps for Firefly III that can be easily used for this using a robust OAuth or shared token authentification. Except the web UI needs to be exposed online for this to work (and ideally protected by HTTPS). Other users have also created Telegram bots that allow using chat messages to create transaction entries. This is a bit harder to use and more narrow scope, but should be easier to setup. Will have to try both apporaches.

But before I really get going on this, I need to fix another thing that I have been postponing for a while: I will need to migrate my Home Assistant installation from a Docker container installation to a Home Assistant OS installation so that I can install Addons, including Firefly III, MySQL and Portainer to have a bit more organised and less hand-knitted home server setup. Let's see how that goes next week :D

I have been managing my finances and getting an overview of where I am financially and where I am going month-to-month for a few years already. That means that I already have a method of doing my finances and a method of thinking about them. So far this has been supported by a commerical tool called MoneyWiz. I was happy to pay them for the ability to have a solid product and be able to easily access my finances from my phone (to enter cash transactions directly in the field) and sync data across multiple devices with their cloud offering. However, MoneyWiz development stalled. So much so that they stopped updating their Android app and cancelled web version plans to just focus on a iPhone and MacOS clients. And even there the development did not seem very successful over the last year. So I am cancelling their subscription, extracting all my data (as CSV) and looking for alternatives.

So let's start with the basics - what I want from finace software?

• Open source and self-hosted
• Accounts to represent real accounts in my bank, credit cards, cash account
• Transactions to represent money moving in/out/between accounts
• Tree of categories for transactions to categorise spending
• Category reports on arbitrary time periods
• Ability to see the expected balance on each account at any historic point in time from given transactions
• Ability to do a corrective transaction - hmm, I don't know what happened, but right now I have X€ in cash
• Adding past transactions before a corrective transaction should not change balance after correction
• Ability to automatically import transactions from my bank
• Ability to plan future, recurring transactions
• Incoming transactions from bank import should be auto-matched with planned recuring transactions
• Ability to see missed planned recurring transactions
• Ability to see value deviations in planned recuring transactions
• Ability to manually match a transaction with a planned recurring transaction for that month or skip a month
• Ability to see a projected balance on my accounts at any future date given planned transactions
• Credit card fully refilling its current (negative) balance to zero (from a given account) should be a plannable transaction
• Accessible from multiple devices, like Linux, Android, iPhone, MacOS. Most likely that means web-based
• Achiving accounts without loosing their past transactions with active accounts
• API to access data, so that I could wire that up to a Home Assistant dashboard

I don't care about currencies (thanks Euro!), localization, taxes and double-entry bookkeeping. I don't have much use for tags, budgets or savings targets.

It could be useful to be able to track stock investment values (but I do have a pretty good from the bank that does that anyway). And tracking loans (incoming and outgoing) could be a worthwhile thing as well, but that can also be simulated with separate accounts for each loan. Tracking refundable expenses (like medical expenses that insurances should refund later on) would also be nice. Could also be simulated by having a separate expense category (Medical - refundable) and putting the refund as a negative expense into that category.

One weird feature that I really like is having transfer transactions that have arbitrary incoming and outgoing dates. For some transactions it is a simple transfer delay - money leaves the account A on Friday and only appears in the destination account B on Monday morning. However, this can also happen the other way around - a few credit cards I have seen work in a way that on 14th of each month the balance of the credit card gets reset to zero and then a week later the negative balance that was on that credit card gets taken from the base account that the card is bound to. So the money leaves the base account a week later than it arrives in the credit card account. Financial systems really are magic sometimes :D

Ideally that should be represented by a single transaction with two dates - one date when money leaves one account and another date when it arrives in a different account, so that balances on both accounts in the days between would be correct (as in matching what the bank says they were on those dates). And it should automatch such transaction from bank data imports. And predict its value from the negative balance of the credit card. And a pony! But in worst case this could be simulated using a separate "In transfer" account.

When I am thinking about my finances, the key metric for me is - how much will I spend this month - both in fixed spending (rent, Internet and phone bills, health insurance, subscriptions, ...) and in variable spending (groceries, eating out, clothing, ...). In theory that should be trivial, but in practice a simple calendar month view will fail because a bunch of fixed expenses occur at the month end/start boundary and can happen either at the end of previous month or at start on next month, depending on how the weekends happen to fall and how the systems of various providers decided to do direct debits this month. So 1st of month is not a good snapshoting time. Any date between 6th (last possible days for monthly bills) and 20th (earliest possible salary day) would be a better cut off point. Looking at the balance of my main account at that time point lets me know what was the difference between income and expenses for the past month. And that is the number I want the financial app to be able to predict as soon as possible.

And additional complication is the way credit cards work here. The expenses booked to a credit card after 14th of September will only appear in the main account on 20th of October and thus will actually only count towards the expenses of month of October. That makes it very confusing to see a larger overspend in the main account on 6th of October and then use the categories to try to figure out where the money went, because all the spending that happened on the credit card after the 14th of September should not really be looked at in that context, but spending on the main account or cash account should be counted all the way to 6th of October. :D

I am not optimistic that any finance software would able to doea with this mess correctly.

So far I am tending to consider Firefly III which has most of what I need and looks great and well maintained, but has the tiny drawback of being written in PHP, so that basically excludes me from ability to contribute anything beyond ideas and feedback. :D

I already did a quick test of Firefly via local Docker containers, so the next step would be to try to set it up as a production instance (using the same always-on mini PC that is running my Home Assistant instance), get the database up and working, get the Firefly III itself up and running, get account importer working for bank connection, import historical transactions from there, setup my categories and recurring transactions, import past data dumped out of MoneyWiz and see if this works well and gives me the insights I need.

If you know a better solution, please let me know on any social or email channel!