• Paperless Extractor

    When I was designing ReceiptWallet, I was concerned about the database being corrupted and losing all my data. I decided that as a backup to the database, I'd store all the files as PDFs (later in life it would store other types of files, but for the most part people only stored PDFs) and would embed all the metadata as PDF keywords that could be recovered. The keywords in the PDFs also served as a transport mechanism where you could send a PDF from ReceiptWallet to someone else with ReceiptWallet and when imported, all the metadata would be filled out. In addition to being used as a backup of the data, I envisioned the day when I no longer used ReceiptWallet as I didn't want to keep supporting it.

    While this original idea seemed pretty solid, writing the keywords for the PDFs proved to be somewhat problematic. At the time, Apple's PDFKit wasn't completely reliable and could corrupt files when writing them out or crash. Reluctantly I added a switch to ReceiptWallet to turn off this feature (default was to have this on). So depending on the PDF, some PDFs may not have keyword data in them so metadata would basically be lost.

    As I wrote earlier this year, that day has come. In my transition away from Paperless (formerly ReceiptWallet), I manually dragged out all files from one of my libraries into the file system and organized them into folders by categories. I placed the folder (named Document Library) in iCloud Drive allowing me to access all the files from anywhere including on my phone. At one point in the past, I started looking at an iPhone version of ReceiptWallet that would sync files; it didn't go anywhere. Now I finally have that feature!

    While I moved a document library over to the file system, I left all my libraries for receipts (one for each year) in Paperless and left them to deal with another day. A few weeks ago I was talking to my father about his transition away from Paperless (he likes to follow my lead sometimes) and I asked him if he had a tool to extract all his files, if he'd use it.

    I spent a few hours that day working on a small Mac application that did just that. I had a chance to do a little SwiftUI and Swift concurrency. You drop a Paperless library onto the app's main window, it walks the library (the library is a bundle which is just a folder which a file extension) looking for every PDF and then opens each file to find the date and merchant. It then creates folders by year and month for the files.

    I've made the source to this app freely available. It comes with no warranty and I will not make a binary of it available; the tool has worked fairly well for my father and me, but may have some quirks (at one point, extra PDFs without metadata were added to the Paperless libraries).

    This app was made without using any source to ReceiptWallet or Paperless; the directory structure of a library can be viewed in the Finder by control clicking the library and choosing Show Package Contents.

    I have no information on the future of Paperless and I'm switching away from it as the file system handles my needs better than it did when I wrote ReceiptWallet over 16 years ago.

  • Goodbye old friend, ReceiptWallet

    About 16 years ago, I was searching for a receipt and couldn’t find it. I thought there could be a better solution and searched for an application for the Mac that would allow me to store receipts. Unfortunately, at the time I couldn’t find anything so I decided to write ReceiptWallet.

    I made many decisions writing ReceiptWallet that turned out to be very forward thinking that would allow me to switch how I tracked receipts in the future. I decided on using PDFs for storing everything (later I allowed for storing zip files and some graphics formats) and storing the files in the file system in a file bundle instead of embedding them in a proprietary database. In addition to storing metadata in a database, I added it to the PDFs that could be extracted without running my software. This was problematic as sometimes the PDFs that were imported either didn’t accept being modified with the metadata or the PDFKit APIs would crash.

    ReceiptWallet was quite successful for me, but other things (like being a father) made it hard for me to keep up with it as a side project. So I decided to sell it and it became Paperless. Over the years I’ve done some work to help them with the product.

    At the end of last year I was looking at Paperless as I used it daily and started thinking how long it would be around. It hasn’t made it to native Apple Silicon, yet and the user interface is becoming very dated. After chatting with a friend that said he just uses Spotlight to find things, I started looking to see if I could recreate the main features that I used in Paperless in the Finder. I also looked at some other apps that were/are competitors to Paperless and didn’t quite find what I wanted. I used the software for two distinct types of filing; the first was for manuals and documents. The second was for receipts and taxes.

    For the first type of filing, I had already started copying certain documents to iCloud Drive such as manuals, certifications, vaccine records, etc. so that I could access them when not at my computer (I had started work on an iPhone version of ReceiptWallet that synced, but didn’t make much progress on it as storing all those documents on the iPhone would quickly eat up the limited storage at the time). I went ahead and made a directory called “Document Library” and then put folders in it. I then transferred all the documents out of my Paperless document library into folders and sub folders in the filing system. It took a few hours, but I managed to classify all my documents. By placing the new “Document Library” on iCloud Drive, I immediately gained mobile access without a special application. I could have used Finder tags, but decided that the directories and searching would be more than adequate. One concern is backing up iCloud Drive as it doesn’t directly appear in the file system. Luckily it is there under ~/Library/Mobile Documents/com~apple~CloudDocs and can be backed up with Carbon Copy Cloner or other tools. Note, however, that if you use tags and backup files to network volumes, CCC by default won’t copy the tags.

    Documents Folder

    For the second type of filing, receipts, I looked at what I was doing in Paperless. I organized receipts by year and then added categories to them in the interface. In reality, the only reason to add the categories was in case I needed to find a receipt for tax purposes. So I setup a folder called Receipts, created a 2023 folder and then a folder for each month under it. Not strictly necessary, but it is easier for me to open a folder with maybe 25-30 receipts than one with 500. Now all I do is scan or save receipts to the right month and I’m done. I don’t worry about filling in all the fields that I did with Paperless such as category, payment method, or amount. Since the folder is in iCloud Drive, I could just snap a picture of the receipt from my phone, save it and be done. I don’t do that, however, as I like the better quality of a scan, my scanner software does OCR, and I still enter every transaction in Banktivity. I do apply tags to certain transactions that are business or tax related which is kind of handy and the tags are visible on my iPad.

    Receipts Folder

    Am I going to miss Paperless? It does seem like an end of an era for me, but by simplifying how I handle receipts and documents will hopefully lead to time savings in the future.

  • Money Donation Machine - My son's Eagle Scout Service Project

    [As this article relates to my son's Eagle Scout service project, I wrote most of it, but he had editorial discretion and added some content in various places. In the article, "I" refers to Scott, and "he" refers to Aiden, my son.]

    About 9 years ago, my wife and I decided that we wanted to impart a sense of giving back to our son, so we decided to start volunteering with Feeding San Diego. We've been regular volunteers since, and when the opportunity arose to become trained as Team Leaders, we did so and our son, Aiden, at age 10 became the youngest team leader.

    When it came time in August 2020 for Aiden to start planning his Eagle Scout service project, he wanted to do something for Feeding San Diego. He brainstormed with their marketing department, and they decided on a cash donation machine similar to ones at science museums and children's museums, which suck the money up through a series of clear tubes. Aiden went back and forth with them over the course of many months to come up with a design and get it approved. While ironing out the design, he started figuring out the actual mechanics of how it would work. I was as hands off on the project as possible, but gave some suggestions here and there. While this donation machine won't be handling million dollar donations, it will hopefully encourage people to donate, and be used as a talking piece in their lobby.

    Finished project with graphics

    He started his concept using an open source project he found, but it didn't quite meet his needs. The first part was to be able to turn on a vacuum when some type of trigger (more on that later) was activated. I prototyped using a WiFi outlet and a WEMOS D1 Mini, but didn't tell him how I did it just to give myself confidence that it could be done. It was a pretty simple concept where I setup the WEMOS as a WiFi access point and had the outlet connect to it; when a button was pressed, the WEMOS (basically an Arduino) sent an HTTP command to the outlet running Tasmota firmware. This concept was great, but in the end there was too much wireless interference when Aiden installed the project for it to actually work. Luckily Aiden found a UL listed wired switch called the Dataprobe iBoot-IO. This product was quite expensive, but turned out to be reliable and met our requirement of being UL listed.

    After getting the outlet to trigger, the next piece was how should it trigger? Basically, when money is put in the tube, it breaks an infrared beam and sends a signal to the outlet. This was completely out of my scope of knowledge, so Aiden did some research, figured out how to wire an IR sensor and receiver. The schematic is below. With that out of the way, how was it going to be mounted? Aiden has become a pro at using Fusion 360 and designed a ring to house the electronics and fit around the tubing. As part of programming the sensors, Aiden wrote the Arduino code to turn off the outlet after a certain number of seconds to allow the money to go through the series of tubes.

    IR sensor

    Is the project done, yet? I kept asking myself that numerous times over the 10 months of the project! Feeding San Diego wanted to make the cabinet to house the vacuum, electronics and collect the money, so Aiden designed and provided the dimensions for the box (again using Fusion 360). Unfortunately, the box wasn't made to those dimensions, but that turned out to be a good thing. The box had a compartment for the vacuum and then a top compartment where the money would go that had a window for people to see. Aiden's assumption on the box is that the top part would be completely sealed such that the vacuum could be attached to the bottom and there would be enough suction to draw the money through the tubes. I wasn't, however, convinced of this and suggested that Aiden have a backup plan. I gave him an idea of a flap on the end of the tube that would close when the vacuum was turned on (the vacuum would provide enough suction to close the flap). Unfortunately the vacuum didn't do that, so Aiden expanded on the idea and designed a flap with a servo such that his program sent a command to the servo to close the flap when the vacuum was turned on and then opened the flap when the vacuum was turned off so that the money would just drop.

    Suction flap

    With that finished, Aiden was confident that everything would work when installed. A few days before he installed the project, we were able to pick up the box so that Aiden could prepare it by installing thick foam to deaden the sound of the vacuum. When install day came around, Aiden led a number of Scouts and adults from his troop on attaching it to the wall. At the end when it came time to turn on the machine, the wireless interference I mentioned before caused problems with it not being reliable. After trying to troubleshoot it, Aiden finally found the iBoot IO box and ordered it as that would be more reliable and he wouldn't have to worry about it. Waiting for the part, however, was torturous for him.

    iBoot IO Box

    Once the part arrived, we went back to Feeding San Diego and he installed it. The electronics worked well and did what it was supposed to do. Unfortunately, there was another hiccup and that was that if the money wasn't crumpled, it wouldn't make it through the tubes. Back to Fusion 360 to design a "cap" to make the hole smaller and require people to crumple the money (signage has been added). Was that the final piece in making everything work? No. Turns out the vacuum was a bit underpowered, and the money wouldn't always make it through even if it was crumpled. More research and a trip to Home Depot for a larger vacuum got him one step closer. Remember I mentioned the larger box before? The new vacuum was larger and some of the insulation he put in had to be cut out to accommodate it.

    Was he done after the new vacuum? Well, almost. Aiden didn't follow the KISS principle in his code which caused some false activations on the vacuum. I encouraged him to remove extra code and just turn on and off the outlet when the sensor was tripped; he had code in there that attempted to detect if someone kept triggering the vacuum and was playing around. That code, unfortunately, was a bit too complex for this application.

    Does it work? Yes!

    Aiden wanted me to write up his project and I've done my best to capture the big pieces.

    If you have any questions about this, please don't hesitate to contact me.

    Complete Project Picture

  • Website Monitoring using Home Assistant and Node-RED

    Recently I was asked by a Scout unit to try to let them know when an event was available for signup. While there are sites that allow you to do this, I didn't want to have to signup for another site so I decided to figure out how to get Home Assistant to notify me of changes. I've written in the past about using Node-RED with Home Assistant and while Home Assistant has made it easier to create automations without having to use YAML, I still find that Node-RED is a much cleaner and simpler way of creating and viewing automations.

    So I started trying to figure out how to get Node-RED to check websites and notify me. Turns out the process was pretty easy.

    1. Add node-red-contrib-object-hash to Node-RED's configuration.
    2. Tell Node-RED to do an HTTP GET on each site once an hour.
    3. Take the result of the HTTP GET and perform a hash on the results.
    4. Create a Home Assistant sensor using MQTT with the URL and the hash in it.
    5. Wait for a change in the hash and then notify me of the change (I use Slack right now).

    It's a pretty simple slow and the downsides are that there is a slim chance that a change would trigger a hash change (hash collisions are possible, but rare) and it doesn't tell me what the change actually is.

    I'm sure there are other ways to monitor sites, but I don't have to rely on others for this. While the notification uses Slack, I can switch out the notification to any other method.

    Picture of Node-RED Flow

    You can download the flow here.