Question for the audience: Do you tell people you blog?

Yes… No… It depend. It is not a thing I talk about my blog in my every day life. And I have more than one blog so…

So I tell people about my blog?

Yes I tell but not unprompted. I told people while talking about the argument of my blog, like if I have a post about what we are talking or if the topic of the blog can be usefull for the discussion.

Am I anonymous when blogging?

I appreciate some people like to be anonymous when blogging. That’s their choice. I am an open book. You can ask me anything and I will reply honestly, as long as it conforms to my town-square rule.

In my case is not so black and white. This blog url and title is Fundor333. This is my online alias. If you find somewhere fundor333 you find one of my account. So I am well know as Fundor333. So I am “anonymous” when I blog (as I don’t use my real name) but I don’t hide my alias.

Conclusion

So I tell people I blog but not at random and I don’t hide my blog. But is not one of my opening line when talking with someone.

The last one I own is a Kobo Clara Colour, this one.

I love it and the fact I can read in color comics, manga, web comics and graphic novel everywhere is a plus.

But more and more I use Ebook Readers more and more I want to customize it. Like I have in the pic.

What can I do? What do I want to do with my Kobo?

I have some task and prerequisite for my Ebook Reader so I check an Hack which resolve all:

• I need to sync with Calibre: I have an huge Calibre Library and I want to keep it
• Change the default cover: I want my cover, not the default covers
• Integration with GoodRead or Hardcover: I want to sync with GoodRead or Hardover the data from my ebook reader.
• Open Source software: I want to use an Open Source project because I can code so if i need something I can code it.

So I did some research on the web and find KoReader¹.

It is a software which, after installing on the ereader, it add support for plugins and custom covers… So I start with it.

How to install it?

Searching for the wiki page for starting/installing I found some tread about the installation² and I follow them. Here I rewrite all the steps with newer link for the stuff needed and rewrote some step because the two treads aren’t clear enough for me.

0. Check if your device support the software
1. Download the last version of what you want to install. In this case Last version of KoReader   is what I want and support on my device
2. Put all the zip/archive you download in a new directory without unarchive anything
3. Download the installer script (Windows version   , Mac version   , Linux version   ) and put in the same directory as all the other stuff downloaded in 2
4. Plug in the Kobo and check if you can read and write file inside it
5. If needed fix the permission for execute the installer download at step 3
6. Follow the on-screen instruction and close the terminal when done
7. Safely eject your Kobo, watch it process (it look like in some way a glitching and crashing device but it need to look like that) and waite for the end of the reboot.

If you have doubt, here is a screenshot of what the terminal look like in step 6 if you did all in the right way.

After the reboot of the device I start to add the plugins for some customization

Adding stuff

I add some KoReader Plugin, which you need to install.

All the plugin are installed in the same way:

1. You need to download the plugin
2. You need to extract the plugin_name.koplugin
3. Edit any config file for the plugin (only some plugin required this)
4. Put the koplugin folder in the path for the device (in Kobo case is .adds/koreader/plugins)
5. Restart KOReader

• ProjectTitle KOReader   for a better looking homepage for KOReader (needed)
• Customisable Sleep Screen   for having a customisable sleep screen (as the name of the plugin say)
• Hardcover.app for KOReader   for sync the data with my HardCover   account

I also create some image with trasparent background for having this type of cover, where you can see under the sleep-screen what are you reading.

And I did more

I did a lot of configuration inside of KOReader for this or that component and I don’t want to lose all. So, after asking Reddit How can I make a backup?   and make this script.

#!/usr/bin/env python3

import os
import sys
import shutil
from datetime import datetime

KOREADER_DIR = ".adds"
BACKUP_DESTINATION = "~/BackUp/KoReader"
BACKUP_FILENAME = "koreader_backup_{date}.zip"


def find_kobo_device():
    """Find the path to the Kobo"""
    volumes_path = "/Volumes"

    if os.path.exists(volumes_path):
        for entry in os.listdir(volumes_path):
            full_path = os.path.join(volumes_path, entry)
            if os.path.isdir(full_path):
                try:
                    contents = os.listdir(full_path)
                    if ".kobo" in contents:
                        return full_path
                except PermissionError:
                    continue

    return None


def get_koreader_path(kobo_path):
    """Returns the path to KoReader on the Kobo."""
    return os.path.join(kobo_path, KOREADER_DIR)


def create_backup(source_path, backup_destination):
    """Creates a zip backup of KoReader."""
    if not os.path.exists(source_path):
        print(f"Error: KoReader not found in {source_path}")
        return False

    print(f"Creating backup from: {source_path}")

    date_str = datetime.now().strftime("%Y%m%d_%H%M%S")
    backup_filename = BACKUP_FILENAME.format(date=date_str)
    backup_path = os.path.join(backup_destination, backup_filename)

    print(f"Backup to: {backup_path}")

    try:
        shutil.make_archive(
            backup_path.replace(".zip", ""), "zip", root_dir=source_path, base_dir="."
        )
        print(f"Backup created: {backup_path}")
        return True
    except Exception as e:
        print(f"Error during backup creation: {e}")
        return False


def main():
    kobo_path = find_kobo_device()

    if not kobo_path:
        print("Kobo not found. Connect the device and try again.")
        sys.exit(1)

    print(f"Kobo found at: {kobo_path}")

    koreader_path = get_koreader_path(kobo_path)
    backup_destination = os.path.expanduser(BACKUP_DESTINATION)
    os.makedirs(backup_destination, exist_ok=True)

    success = create_backup(koreader_path, backup_destination)

    if success:
        print("Backup completed successfully!")
    else:
        print("Backup failed.")
        sys.exit(1)


if __name__ == "__main__":
    main()

This Python script is run every time I connect to my computer with the Kobo and copy the .adds in a specific path with a new folder tagged with the timestamp.

So if I break my KOReader config, I can reset all and copy the .adds for restore the KOReader.

Conclusion

For now this is what I did with my Kobo but I am thinking new stuff so if I will do something new I will write about it.

No he didn’t work as I wanted so I did some editing of the code here and there.

The old one

We start with the code I didn’t change:

First the model

# blog/model.py

class Post(models.Model):
    title = models.CharField(max_length=400)
    slug = models.SlugField(max_length=400, unique=True, blank=True)
    content = models.TextAreaField()
    date = models.DateTimeField(auto_now_add=True)

The table

# blog/tables.py

import django_tables2 as tables
from blog.models import Post

class PostTable(tables.Table):
    class Meta:
        model = Post
        order_by = ("-date", "title")
        fields = ["date", "title", "slug"]

This two fragment are perfect as is so I didn’t touch them. But all the other part I reworked as I needed.

The new parts

I need to have a “universal table” for show data, with filter and export the data. And I need to have an “universal template” for all the combination Table-Export-Filter…

So I wrote one

First thing (after the model and the table) I need view with filter and exporter support. And using django-table2¹ all was easy.²

So I wrote this class which support all I need for using it as Htmx endpoint.

# blog/views.py

from django_tables2 import SingleTableMixin
from django_tables2.export import ExportMixin, TableExport
from django_filters.views import FilterView
from blog.model import Post

class DataHtmx(ExportMixin, SingleTableMixin, FilterView):
    model = Post
    template_name = "generic/table2.html"

    table_class = PostTable

    filterset_fields = ["title","slug", "date"]

    export_formats = (
        TableExport.CSV,
        TableExport.XLS,
        TableExport.XLSX,
        TableExport.ODS,
    )

    def get_export_filename(self, export_format):
        return f"Export Data.{export_format}"

• ExportMixin, with the export_formats are the the export part of the view
• SingleTableMixin is the table part of the view
• FilterView is the filter part of the view, with filterset_fields as the field you can filter. You can also haave a filterset_class if you need

Done this I need a template. More generic template because I don’t want to rewrite the same code every time…

So here is it!

# templates/generic/table.html

{% load render_table from django_tables2 %}
{% load export_url from django_tables2 %}
{% load django_bootstrap5 %}

{% if filter  %}
  {% if 'hide_filter' not in request.GET %}
      <div class="col-12 col-md-9">
        <div class="card mb-4">
          <div class="card-header">
            <button class="btn btn-section" type="button" data-bs-toggle="collapse" data-bs-target="#filterCollapse">
              <i class="fa-solid fa-filter"></i> Filtri <i class="fa-solid fa-chevron-down"></i>
            </button>
          </div>
          <div id="filterCollapse" class="collapse">
            <div class="card-body">
              <form method="get" class="row">
                {% load django_bootstrap5 %}
                {% bootstrap_form filter.form layout='floating' %}
                  {% include 'generic/_btn_search.html' %}
              </form>
            </div>
          </div>
        </div>
      </div>
{% endif %}
{% endif %}

<div class="row">
    {% if 'hide_export' not in request.GET %}
      {% for format in view.export_formats %}
      <a  class="btn col col-sm" href="{{ request.path }}{% export_url format %}" target="_blank">
        <i class="fa-duotone fa-regular fa-download"></i> <code>{{ format }}</code>
      </a>
      {% endfor %}
    {% endif %}
    <hr>
    {% render_table table %}
</div>

This template work if you have filters, export or not.

• the load are for django-table2 and boostrap (the last is for me, is not needed for the code)
• ‘hide_filter’ not in request.GET check if the url has the params for hide_filter. If it is present, the template hides the filters
• ‘hide_export’ not in request.GET check if the url has the params for hide_export. If it is present, the template hides the exports

One example for the Htmx code is the following.

<div hx-get="{% url 'blog:post_htmx' %}?date_at={{ query_date | date:"Y-m-d"  }}&hide_filter" hx-trigger="load">

In this example I had the filter set (with data form the template) and with hide_filter which hide the filters but not the exports button.

I thing this will be a stable of my django code and it can be move in a custom module for share and easy import.

Xkcd - Virtual Assistant

If you ask it to please turn off that feature, it apologizes a whole bunch and promises to try to be quieter, then switches to a slightly lower-volume version of the clip with "sorry!" after the louder sounds.

What do I need?

First I search for understand what I need to have for build my personal home automation. So serching on the web I found I need:

• A brain/core/server for coordinate all the stuffs
• A software for the brain, for the automation and other stuff
• A router for adding the brain to the network for update and other stuff
• Dongle/sensor/tech for connectivity with the sensors and smart stuff (one for protocol)
• The smart stuff (light bulb, sensor, timer, endpoint api…)

So I need to choose which protocol I want in my home… And I don’t know…

So I change the point of view:

• I want to use good smart stuff but I don’t want to pay too much
• I want something Open Source so I am not bind to a company
• I want something offline or/and on MY HOME SERVER

The protocols

Searching for the protocols I choose Thread² and Zigbee³. One reason is because they are the most used protocol for communication and more easy to find online (Amazon) and offline (Ikea and othe local shop).

For starting I will implement only Thread because the more easy tech for me are all Matter over Thread so I start with Thread.

From the infographic I found, I need to implement Matter⁴ also because Matter is the top stuff for an Home Assistant.

The software

Now I want a software, Open Source, which I can support Thread and Matter (and also Zigbee) and I found a software (Open Source) wich support all I need: Home Assistant⁵.

It support Matter⁶, Thread⁷ and Zigbee⁸ and I can put it in a RaspberryPi or other server.

Hardware

So I start to make a list of what I have and what I need. Some things I buy online, some things come from some tech market and some come form the shoes box of my old stuff…

I only write about the real things but if I use cable or other “standard stuff” I don’t write about it. So yes I bought some usb extention and some network cable but they aren’t important for this post.

Core/Brain Stuff

First I need the Brain for the project

From what I read in the Home Assistant⁵ documentation I need some type of server with some gadget connected to them.

So I search and got a RaspberryPi 4B⁹.

I also bought a microSD with 64GB of memory (I am on a budget for this project) as an Hard Disk for the “server”.

Searching in my closet and in some shoes boxes I found a case¹⁰ for RaspberryPi with active cooling system (and LGB light)

So I flash the microSD with the Home Assistant software. Thanks to the Raspberry Pi Software¹¹ choosing the right iso and flash it in the SD was a walk in the park.

After the flashing I set it up and add to my local network and check for update.

So the brain is ready!

Sensor form Ikea

I went to Ikea for some stuff and some Swedish meatballs and I came back with an Ikea Alpstuga¹².

It is a timer, air sensor with CO2 and PM2.5 sensor and output all of them and an Air Quality value.

The only bad thing about it is the color but for the price I don’t complain.

So after coming home I found it didn’t work.

As is the RaspberryPi 4B do not support Thread or Zigbee without some type of specialized antenna… I was understanding that the software use the wifi module of the RaspberryPi as a Zigbee/Thread emitter/reciver but no so I search on Amazon for the dongle for Thread.

Antenna

I bought a Usb dongle¹³ for connecting the RaspberryPi and the other gadget in the house (for now only the Ikea one)

¹³

This antenna is a little special: you can flash it for changing it form Zigbee to Thread and vice versa so I start to config it.

Setup the Antenna

First I flash the antenna. I need to port from Zigbee to Thread¹⁴ and with the online tool it was so easy I thought I did something wrong.

After this I connect the dongle and start to config the Home Assistant.

First I need to have installed Thread from “Add Integration” and it showed an empty list of device. This is correct because I didn’t have a Thread router installed.

So we add a OpenRouter. How? Installing from the officials app of mine Home Assistant.

After this I edit the config in this way (the device was auto-finded) and I set this for my device. ¹⁵

For checking if all was ok I went and seach in the log for the OpenThread for a specific line.

After this I went to check if the Thread integration showed the Ikea sensor and it didn’t show… Why? Because I am STUPID.

Setup the last little things

Alpstuga (my air sensor) is not Thread, is Matter over Thread, so I need to add the thing with the Matter integration. And with this last integration the system worked!

I am happy of this system? No. Why? I have the wrong time on the Alpstuga.

This thing I don’t understand so I search online for understand how can I fix the time on the device.

I find this is a common problem with the Alpstuga device and someone made a Hacs¹⁶ plugin¹⁷ wich add a cron and a button for sync the Home Assistant’s clock. I also add the plugin for integrate my Playstation Network in my Home Assistant¹⁸.

Conclusion

For now this is what I had done but I want to have automation (for now I don’t have any) and some stuff for lower the gas bill so there will be more post about my smart home.

Some of the arguments are old, some are new but all come from my dev stack.

Build a .venv

For every Python project I wrote, I code all the project with a Virtual Env or .venv. It is a local “installation” of Python for the folder and it keep all the dependency inside it.

It is the best way to develop python project because you can have any number of Python project, with different version of Python and/or different version of the dipendency for each project.

Because I use git I need a good way to lock and commit the dipendency of my project. Some time ago I was using Poetry but now I use UV for build, update and manage the Virtual Env

Creating a .env file

Every python project I create, I always add a .env file, where I store all the environment variable for the project but I don’t add them to the repo because It is not necessary add the local config for dev in a repo.

You can also automatically use the .env file in Uv so every time you are inside the .venv, you also have all the variables.

Using UV

So after some time I was using Poetry and Pipenv, I find UV   , a tool write in Rust for locking dipendnecy, set up and manage the .venv and create package for the publication in one tool.

You can easly start a procjet with a simple command

uv init

which create the pyproject.toml and the .venv with all the dipendency.

With some other command you can easily add, remove and update the dipendency or import in a easy way your .env file.

uv run --env-file=.env

I move to UV because I can use a single tool for all my python necessity and it is faster and less bugged.

Editor Config

We have the .venv, we have the .env, we have the dependency manager so I add the Editor Config file for the style of the code. The Editor Config file   is a file where you define the style for the type of file you write in your repo.

I have mine made year after year but I start with the example   in the site because it is easy and with all the stuff you need.

Makefile it

So, after adding stuff over stuff to the repo, I add a makefile where yu have all the aliases for a python project, ¹.

My basic makefile is something like this

SHELL := /bin/bash

RUNNER := uv run --env-file=.env

.PHONY: help
help: ## Show this help
	@egrep -h '\s##\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

.PHONY: install
install: ## Make venv and install requirements
	@uv sync
	@$(RUNNER) pre-commit install
	@pre-commit autoupdate

.PHONY: update
update: ## Update requirements
	@uv lock --upgrade
	@$(RUNNER) pre-commit autoupdate

precommit: ## Run pre-commit hooks
	@git add . & uv run --env-file=.env pre-commit run --all-files

The help command come form a blog post² and describe better at my other post³

I write a makefile for all my project because I forgot a lot of the command and parameters, so a makefile is a good way to have an “deploy” command or a “test” command for the project and forgot all the task needed for run a “test” or a “deploy”.

I also put all the test command into the makefile because sometime the test command is too long to remember…

Pre commit and some automation

This is my personal safety net for the commits. Some time I wrote and push without any work or thing about what I am doing… For this reason I add pre-commit⁴ for autofix and check all this minor error.

The took launch a lot of test BEFORE I can save the commit so I can find the error or getting a big fat NOPE for the commit, which I need to review for the problem.

Conclusion

This is the stack I work with and I love it. It can be something custom and old style for some of the older tecnology but I find it complete for what I do.

What I have

I have a MonthArchiveView with render a table list with a lot of function I need: add element, change month show, exports query as files, etc…

I also don’t like to reload all the page for change the month I am seeing on the page. So I start to implement what I need.

And, from some comment from the web and some friends, I want less code and more clean one.

What I used

For this project I used:

• Django the framework used¹
• Django Table2 a module for easy creation of table for Django²
• Django Htmx a module for easy support of Htmx³
• Tablib a module for exporting the table as multiple file type⁴

The Py code

So I start with the code for the model. It need to have a DateTimeField or a DateField (this becose I want to have an MonthArchiveView view).

# blog/model.py

class Post(models.Model):
    title = models.CharField(max_length=400)
    slug = models.SlugField(max_length=400, unique=True, blank=True)
    content = models.TextAreaField()
    date = models.DateTimeField(auto_now_add=True)

and after I add the code for the table with the ordering of the data

# blog/tables.py

import django_tables2 as tables
from blog.models import Post

class PostTable(tables.Table):
    class Meta:
        model = Post
        order_by = ("-date", "title")
        fields = ["date", "title", "slug"]

After the model and the table we need the view called by the HTMX code.

I set all the config for choose the right data field and which is the format for month and year, followed for the export types’ settings.

# blog/views.py

from django.views.generic import ListView
from django.contrib.auth.mixins import LoginRequiredMixin
from django_tables2 import SingleTableView, tables
from blog.models import Post
from blog.tables import PostTable

class PostMonthArchiveHtmx(LoginRequiredMixin, ExportMixin, SingleTableMixin, MonthArchiveView):
    model = Post
    template_name = "generic/table2_with_export.html"
    table_class = PostTable


    date_field = "date"
    allow_future = True
    month_format = "%m"
    year_format = "%Y"
    allow_empty = True

    export_formats = (
    TableExport.CSV,
    TableExport.XLS,
    TableExport.XLSX,
    TableExport.ODS,)

    def get_month(self):
        try:
            month = super().get_month()
        except Http404:
            month = now().strftime(self.get_month_format())

        return month

    def get_year(self):
        try:
            year = super().get_year()
        except Http404:
            year = now().strftime(self.get_year_format())

        return year

    def get_export_filename(self, export_format):
        return f"Export Post {self.get_year()}-{self.get_month()}.{export_format}"

I added a function for changing the name of the output file (because I like filename with date reference) and two functions for setting month and year if not given (we need them later for the Htmx part).

The Jinja part

After the python code I wrote this generic template for all my django-table2 table (in this case with the export code)

# templates/generic/table2_with_export.html

{% load render_table from django_tables2 %}
{% load export_url from django_tables2 %}

<div>
    {% for format in view.export_formats %}
    <a href="{{ request.path }}{% export_url format %}" target="_blank">
      <code>{{ format }}</code>
    </a>
    {% endfor %}
    <hr>
    {% render_table table %}
</div>

I take out all the css stuff so you can add your style and/or css classes.

At this point we have all we need to have a html page with the table and the exports we need. We only need to add this view into the urls…

For example this is a generic example for a urls file for this project

# blog/urls.py

from django.urls import path

from blog.views import (
    PostMonthArchiveHtmx,
)

app_name = "blog"

urlpatterns = [

    path("htmx/post", PostMonthArchiveHtmx.as_view(month_format="%m",name="htmx-post")),
]

You can see that I didn’t add the code for having the month and the year in the url. This is because I had a lot of problems with the next part and this is the cleanest way I found to code this.

The Htmx fragment

I will assume that you have done the quickstart form Htmx’s site⁵ because I don’t want to write about Htmx…

So for using inside a Django project for calling the page with Htmx we need to add the csrf token inside the header of the request so I added inside the main tag for my project.

It resembles this fragment.

<!-- template/base.html -->

<!-- Other part of the template-->

<div hx-headers='{"x-csrftoken": "{{ csrf_token }}"}'>
  {% block content %}
    (no content)
  {% endblock %}
</div>

<!-- Other part of the template-->

So now I need to write the div for the Htmx. And this is what I wrote:

<div hx-get="{% url 'blog:htmx-post' %}?year={{ month|date:"Y" }}&month={{ month|date:"m" }}&{{ request.GET.urlencode }}"
     hx-trigger="load">This will be replaced.</div>

I build the url in the hx-get in a strange way because I need to add the query input for the MonthArchiveView (the &month and &year field in the url) and add all the other field. In this way I also have the sorting links in the header of the table working.

Conclusion

This is how I have combine this 3 libs for my personal project and if you find it useful share it. Thanks!

The code for the inizializzation

For start I need to install some dipendency

langchain>=0.3.27
langchain-ollama>=0.3.7
pandas>=2.3.2

import pandas as pd
from langchain_ollama import ChatOllama
from typing import Literal

df = pd.read_csv("data/test.csv")

print("-*-" * 20)
print(f"Dataset shape: {df.shape}")
print("-*-" * 20)
print("Missing value stats:")
print(df.isnull().sum())

-*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*-
Dataset shape: (418, 20)
-*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*--*-
Missing value stats:
ID                 0
N_Days             0
Status             0
Drug             106
Age                0
Sex                0
Ascites          106
Hepatomegaly     106
Spiders          106
Edema              0
Bilirubin          0
Cholesterol      134
Albumin            0
Copper           108
Alk_Phos         106
SGOT             106
Tryglicerides    136
Platelets         11
Prothrombin        2
Stage              6
dtype: int64

This is a section of the dataset and what is missing value of the stats.

Now we will start with the AI. In my case I user Ollama with Mistral model. I install the model with

ollama run mistral

And prepare the code for use the model. First you need to make a connection with the local LLM instance. This code use Mistral but you can pass any local LLM instance you have.

def get_llm(model_name: str = "mistral:latest") -> ChatOllama:
    """
    Create and configure a ChatOllama instance for local LLM inference.

    This function initializes a ChatOllama client configured to connect to a
    local Ollama server. The client is set up with deterministic output
    (temperature=0) for consistent responses across multiple calls with the
    same input.

    Parameters
    ----------
    model_name : str, optional
        The name of the Ollama model to use for chat completions.
        Must be a valid model name that is available on the local Ollama
        installation. Default is "mistral:latest".

    Returns
    -------
    ChatOllama
        A configured ChatOllama instance ready for chat completions.
    """
    return ChatOllama(
        model=model_name, base_url="http://localhost:11434", temperature=0
    )

If you want to test the connection you can use this command

print(get_llm().invoke("test").content)

 Hallo! Wie geht es Ihnen? Ich bin hier, um Ihnen zu helfen. Was möchten Sie heute tun?

Ich kann Ihnen beispielsweise helfen:

* Fragen beantworten
* Informationen suchen
* Aufgaben lösen
* und vieles mehr!

Welche Aufgabe haben wir heute vor uns?

Make a context

Now we need to generate a context for the LLM. If you do this function with all the necessary data you can relaunch this script every time you need a new README/summary of the dataset. This is better to be a dataset with a fixed schema and a date which change every year like medical data (this), monthly sell report, census data…

def get_summary_context_message(df: pd.DataFrame, dataset_name:str) -> str:
    # Basic application statistics
    total_analisys = len(df)

    # Gender distribution
    gender_counts = df["Sex"].value_counts()
    male_count = gender_counts.get("M", 0)
    female_count = gender_counts.get("F", 0)

    # Stage Statistics
    stage_data = df["Stage"].dropna()
    stage_avg = stage_data.mean()
    stage_25th = stage_data.quantile(0.25)
    stage_50th = stage_data.quantile(0.50)
    stage_75th = stage_data.quantile(0.75)

    # NDays Statistics
    days_data = df["N_Days"].dropna()
    days_avg = days_data.mean()
    days_25th = days_data.quantile(0.25)
    days_50th = days_data.quantile(0.50)
    days_75th = days_data.quantile(0.75)

    def status_category(exp):
        if pd.isna(exp):
            return "Unkown"
        elif exp == "C":
            return "Censored"
        elif exp == "CL":
            return "Censored due to Lever tx"
        elif exp == "D":
            return "Death"
        else:
            return "Unkow"

    df['Status Str']= df['Status'].apply(status_category)
    status_str_stats = []

    for category in ["Censored", "Censored due to Lever tx", "Death",]:
        category_data = df[df["Status Str"] == category]
        if len(category_data) > 0:
            male = len(category_data[category_data["Sex"] == "M"])
            female = len(category_data[category_data["Sex"] == "F"])
            total = len(category_data)
            rate_m = (male / total) * 100
            rate_f = (female / total) * 100
            status_str_stats.append((category, male, female, total, rate_m, rate_f))

    summary =f"""{dataset_name}

Total Analisys: {total_analisys:,}

Gender Distribution:
- Male applicants: {male_count:,} ({male_count/total_analisys*100:.1f}%)
- Female applicants: {female_count:,} ({female_count/total_analisys*100:.1f}%)

Stage Statistics:
- Average Stage: {stage_avg:.2f}
- 25th percentile: {stage_25th:.2f}
- 50th percentile (median): {stage_50th:.2f}
- 75th percentile: {stage_75th:.2f}

N Day Statistics:
- N Days Stage: {days_avg:.2f}
- 25th percentile: {days_25th:.2f}
- 50th percentile (median): {days_50th:.2f}
- 75th percentile: {days_75th:.2f}
"""

    summary += "\n\nStatus Rates by Sex:"
    for category, male, female, total, rate_m, rate_f in status_str_stats:
        summary += (
            f"\n- {category}: {male}/{total} Male ({rate_m:.1f}% rate)"+
            f"\n- {category}: {female}/{total} Female ({rate_f:.1f}% rate)"

        )
    return summary

Make a report

After checking all you need to have a template for the repo of the dataset.

SUMMARIZE_DATAFRAME_PROMPT = """
You are an expert data analyst and data summarizer.
Your task is to take in complex datasets and return user-friendly descriptions and findings.

You were given this dataset:
- Name: {dataset_name}
- Source: {dataset_source}

This dataset was analyzed in a pipeline before it was given to you.
These are the findings returned by the analysis pipeline:

<context>
{context}
</context>

Based on these findings, write a detailed report in {report_format} format.
Give the report a meaningful title and separate findings into sections with headings and subheadings.
Output only the report in {report_format} and nothing else.

Report:
"""

This prompt and a lot of the code of this article are from this post   .

After this we need a function that take the dataset df, the prompt SUMMARIZE_DATAFRAME_PROMPT with the needed info and return the content of the report.

def get_report_summary(
    dataset: pd.DataFrame,
    dataset_name: str,
    dataset_source: str,
    report_format: Literal["markdown", "html"] = "markdown",
) -> str:
    context_message = get_summary_context_message(df=dataset, dataset_name=dataset_name)
    prompt = SUMMARIZE_DATAFRAME_PROMPT.format(
        dataset_name=dataset_name,
        dataset_source=dataset_source,
        context=context_message,
        report_format=report_format,
    )
    return get_llm().invoke(input=prompt).content

In our case we launch it as

md_report = get_report_summary(
    dataset=df,
    dataset_name="Cirrhosis Patient Survival Prediction",
    dataset_source="https://www.kaggle.com/datasets/joebeachcapital/cirrhosis-patient-survival-prediction/data"
)
print(md_report)

 # Cirrhosis Patient Survival Prediction Analysis Report

## Overview
The dataset analyzed consists of 418 records related to cirrhosis patients, sourced from [Kaggle](https://www.kaggle.com/datasets/joebeachcapital/cirrhosis-patient-survival-prediction/data). The data provides information about the patient's gender, stage of cirrhosis, number of days since diagnosis, and final status (censored or death).

## Demographics
### Gender Distribution
The dataset shows a significant imbalance in gender distribution with 89.5% female applicants (374) and only 10.5% male applicants (44).

## Cirrhosis Stage Statistics
### Average Stage
The average stage of cirrhosis for the analyzed patients is 3.02, indicating a severe level of liver damage.

### Percentiles
- **25th percentile**: The cirrhosis stage is at least 2.00 for 25% of the patients.
- **Median (50th percentile)**: Half of the patients have a cirrhosis stage of 3.00.
- **75th percentile**: For 75% of the patients, the cirrhosis stage is 4.00 or lower.

## N Days Statistics
### N Days Stage
The average number of days since diagnosis for the analyzed patients is 1917.78 days.

### Percentiles
- **25th percentile**: The minimum number of days since diagnosis for 25% of the patients is 1092.75 days.
- **Median (50th percentile)**: Half of the patients have been diagnosed with cirrhosis for at least 1730.00 days.
- **75th percentile**: For 75% of the patients, the number of days since diagnosis is 2613.50 days or less.

## Status Rates by Sex
The following table shows the rates of different statuses (censored due to Lever tx and death) for both male and female applicants:

|                     | Male Applicants | Female Applicants |
|---------------------|-----------------|-------------------|
| Censored            | 17/232 (7.3%)    | 215/232 (92.7%)   |
| Censored due to Lever tx | 3/25 (12.0%)     | 22/25 (88.0%)     |
| Death                | 24/161 (14.9%)   | 137/161 (85.1%)   |

The analysis indicates that female applicants are more likely to have their status censored, either due to the lack of information or other factors, while male applicants are more likely to experience death. However, it's important to note that the sample size for male applicants is significantly smaller than that of female applicants.

And now, after a PyDataVe (the 22nd event), I make a Jupyter Notebook after a lot of time and I feel the need to have a solution for make a Jupyter Notebook one of the post of my blog so I find a way to do it. This time in Hugo

Jupyter in my Hugo

So first I search for a way to convert a Jupyter Notebook to a Markdown file with the headers for Hugo. So I find a python module   to generate a Notebook with a starting metadata for my blog. All the metadata are inside the first cell set as a Markdown type cell.

All this is made by launching this command for generate the notebook

hugo_nbnew content/post/test-hugo-page

And at the end of all the writing I launch this command

hugo_nbconvert content/post/test-hugo-page/index.ipynb

for generate/updating the markdown file for Hugo.

Other little things I have done

After testing all of this stuff I make some script for my personal writing pipeline (if I don’t make some automation I feel bad) I fix some CSS of my site for the code part. I am a Backend Dev, so I don’t share my CSS edit because it is something working but not so pretty to share…

After that I find myself wanting to have some way to cite a part of the post with a link (because I want to share a specific part of the post as a response to post/tweet/tooth on the web) and I search for something clean and usefull.

So I find a JS script   for adding deep anchor links to my post and I set it to add a deep anchor for every paragraph of the posts.

And after all I check if I implement the meta tag for journalism for Mastodon   because I share a lot of stuff on Mastodon.

Final consideration

I like my new configuration with the possibility to add Jupyter Notebook in my blog and this is the first Notebook I had to my Hugo Blog. I also like the idea that I can work for something else, make a Notebook and, only changing the data, make multiple article about something with the updated data every time I need to add the blog with the latest report for the dataset.

Naty S @eclecticpassions@fosstodon.org

New #blogpost

➡️ https://www.burgeonlab.com/blog/hugo-and-wordpress-open-graph-meta-tags/

I often see bloggers on Mastodon with no images for their blog links or sometimes no link card at all (this was me not too long ago).

I've been sorting out Open Graph meta tags for my three blogs (Hugo and WordPress) and I'm happy to say it's all working great now! Read my post to get some tips! 👆

#BurgeonLab
#blog #blogging #blogs #hugo #wordpress #webdev #opengraph

3:18 PM · Jul 8, 2025

and I find I do the same thing and some more. I generate a default cover because I prefer to have a “standard” generated image or a specific image as the feature image. The image set in the settings is a default, a “if all other image fall use this” kind of stuff.

How I generate the post?

When I write a post I launch a makefile command and it launch two command for me:

1. Generate a new post with hugo new
2. Generate a cover with my python script

Some code

For this project I use a font (Futura Book font   ) and a image where put the title and other thing of the post (for now only the title). This is the image “clean”, without the text.

from PIL import Image, ImageDraw, ImageFont  # 👉️ Import modules from PIL


def generate_img(message: str, path: str):
    font_path = "Futura Book font.ttf"  # 👉️ Font .ttf Path
    font_size = 100  # 👉️ Font Size
    img = Image.open("cover-blank.jpg")  # 👉️ Open Image
    dr = ImageDraw.Draw(img)  # 👉️ Create New Image
    my_font = ImageFont.truetype(font_path, font_size)  # 👉️ Initialize Font
    text_x = (img.width) // 2
    text_y = (img.height) // 2
    dr.text((text_x, text_y), message, font=my_font, fill=(255, 255, 255), anchor="mm")
    print("Generated content/" + path + "/cover.png")
    img.save("content/" + path + "/cover.png")

This script use Python (the library is PIL) and I set the fill parameters for a standard size OpenGraph image.

If you use a python script for generate new post you can implement this code inside of the script or you can add some code to make a command line to launch after your “new post” command.

• The date fild of a form with the right input type for the page
• A link with all the get parameters of the current page (form, paginator, etc…) and one added

The Code

This little code is mine but you can use it on all your project without thinking. If you find an error comment or webmention or replay with activity pub to my toot

Date Field for Django Form

This is the widget I add to the django form for adding the right input type.

from django.forms import TextInput

class DateInput(TextInput):
    input_type = "date"

I need this because the css/js framework I use add a data picker if you fill the input_type

Tag for change/keep a field

Sometime I need to edit or add a get parameter in a link, like when you change the view from list to grid (for example). So I coded this simple tag.

from django import template
register = template.Library()

@register.simple_tag
def url_replace(request, field, value):
    dict_ = request.GET.copy()
    dict_[field] = value
    return dict_.urlencode()

the only thing it does is change or add a value into the list of existing value.

For example

<a href:"https://fundor333.com?{% url_replace_diff request 'page' page_obj.page_number %}">Link</a>

In this one I pass the pagination number to the new page. Why? Because I need to have the current page and all the serch filter also in the new page and this is the cleaner way WITHOUT using Js (I need control and Js is not controll)

Robb Knight @robb@social.lol

When I open my RSS reader and it has over 200 unread items, I know one of you has messed up your rss feeds

📌 https://rknight.me/notes/202501190902/

9:04 AM · Jan 19, 2025

Andrea Grandi 🦕 @andreagrandi@mastodon.social

@robb I think my friend @fundor333 ears must be burning right now :D

10:51 AM · Jan 19, 2025

And after read some other post with the same problem ( Robb Knight problem with feed reader   Kev other problem with feed reader   ) I understand a little this: Atom is newer and superior than Rss.

Atom vs Rss

Atom is the newer one, born after the RSS and after fix a lot of the RSS bugs like:

• You have a strict format for checking and validation in Atom
• RSS is simpler but less strict and it is riddled with ambiguities which lead to inconsistency of interpretation
• You have a unique id for the feed in Atom

The last one is the one I was searching…

The Answer

The feed reader find the new feed (new because you change the url or you change the serverside code) and check:

• Atom Feed: the reader find the same ID so is the same feed
• RSS Feed: the reader don’t find the ID and find something different not in the items part so it is a new one

So I add the JsonFeed and AtomFeed to the blog for give the reader multiple option for the feed!

You can find all at this link

And one of the best way to share other blog without filling a webpage with links was the WebRing.

WebRing?

For Wikipedia   a Webring is

A webring (or web ring) is a collection of websites linked together in a circular structure, usually organized around a specific theme, and often educational or social.[1] They were popular in the 1990s and early 2000s, particularly among amateur websites.

Today, with the Search Engin owned by the big tec and the AI being part of the mayor Search Engine we need more than ever list of human web pages to read and less AI generated post.

For example?

Some example of WebRing? This are some type of WebRing

• A themed WebRing Like a WebRing about Python  
• An Inspirational WebRing Like one for dev and artist  
• One for Group of People Like one for onl;y personal site  
• One for a Community Like one about IndieWeb  

If you don’t find a WebRing for your “community”, you can create a new one with a tutorial ( A Webring Kit   ) and a framework for making WebRing ( GitHub Repo from A Webring Kit   )

Remember, this is a free way to have more traffic inside your site and for build your community

The problem

Every code is a solution to a pratical problem. In this case I need to make the barcode labels with text for keep an inventory. The basic solution is generate a pdf/img file to print into sticker’s paper and cut it after.

So, I need the data to convert into lable (the string on top to the barcode) and the barcode (or the qrcode) as in the screenshot, so I wrote this code for do it for pages and ready to print and stick it all around the world!

Code

The view

This code is the view for generating download link for the pdf with all the barcode.

from django.views.generic import View

class ExampleBarcodePdf(View):
    bar_height = None
    position_text_y = None
    filename = "ExampleBarCode.pdf"

    def get(self, request, *args, **kwargs):
      super().get(request, *args, **kwargs)
      response = HttpResponse(content_type="application/pdf")
      response["Content-Disposition"] = (
          'attachment; filename="' + self.filename + '"'
      )
      a4p = A4PrinterBarcode()
      pdf = a4p.get_pdf(
          buffer=BytesIO(),
          datas=[{barcode:"0000000000000", "label"}, {"999999999999", "other label"}],
          bar_height=self.bar_height,
          position_text_y=self.position_text_y,
      )
      response.write(pdf)
      return response

The pdf generator

This is the “printer” the code for generate pdf. After the import you find the constant with define the pdf.ß

from reportlab.graphics.barcode.eanbc import Ean13BarcodeWidget
from reportlab.graphics.shapes import Drawing
from reportlab.graphics.shapes import String
from reportlab.lib.pagesizes import A4
from reportlab.pdfgen import canvas

PAGESIZE = A4
NUM_LABELS_X = 3
NUM_LABELS_Y = 8
BAR_WIDTH = 1.5
BAR_HEIGHT = 51.0
TEXT_Y = 80
BARCODE_Y = 17
FONT_SIZE = 8

LABEL_WIDTH = PAGESIZE[0] / NUM_LABELS_X
LABEL_HEIGHT = PAGESIZE[1] / NUM_LABELS_Y
SHEET_TOP = PAGESIZE[1]


class A4PrinterBarcode:

    def generate_label(
        self,
        ean13: str,
        description: str,
        label_w: float = 0.0,
        label_h: float = 0.0,
        bar_with: float = BAR_WIDTH,
        bar_height: float = BAR_HEIGHT,
        postion_text_y: float = TEXT_Y,
        font_size: int = FONT_SIZE,
    ) -> Drawing:
        """
        Generate a drawing with EAN-13 barcode and descriptive text.
        :param ean13: The EAN-13 Code.
        :type ean13: str
        :param description: Short product description.
        :type description: str
        :return: Drawing with barcode and description
        :rtype: Drawing
        """
        if label_w == 0.0:
            label_w = A4[0] / NUM_LABELS_X
        if label_h == 0.0:
            label_h = A4[1] / NUM_LABELS_Y
        text = String(
            0,
            postion_text_y,
            description,
            fontName="Helvetica",
            fontSize=font_size,
            textAnchor="middle",
        )
        text.x = label_w / 2

        barcode = Ean13BarcodeWidget(ean13)
        barcode.barWidth = bar_with
        barcode.barHeight = bar_height
        _, _, bw, _ = barcode.getBounds()
        barcode.x = (label_w - bw) / 2  # center barcode
        barcode.y = 18  # spacing from label bottom (pt)

        label_drawing = Drawing(label_w, label_h)
        label_drawing.add(text)
        label_drawing.add(barcode)
        return label_drawing

    def get_pdf(
        self,
        buffer,
        datas: list,
        bar_height: float = None,
        position_text_y: float = None,
    ) -> bytes:
        if bar_height is None or bar_height < 0:
            bar_height = BAR_HEIGHT
        if position_text_y is None or position_text_y < 0:
            position_text_y = TEXT_Y
        p = canvas.Canvas(buffer)
        j = 0
        for _ in range(len(datas) // (NUM_LABELS_X * NUM_LABELS_Y) + 1):
            for u in range(0, NUM_LABELS_Y):
                for i in range(0, NUM_LABELS_X):
                    if j < len(datas):
                        data = datas[j]
                        cb = data.barcode
                        desc = data.label
                        try:
                            label = self.generate_label(
                                ean13=cb,
                                description=desc,
                                bar_height=bar_height,
                                postion_text_y=position_text_y,
                            )
                            x = i * LABEL_WIDTH
                            y = SHEET_TOP - LABEL_HEIGHT - u * LABEL_HEIGHT
                            label.drawOn(p, x, y)
                        except AttributeError:
                            pass
                        j += 1
            p.showPage()
        p.save()
        pdf = buffer.getvalue()
        buffer.close()
        return pdf

So how do A4PrinterBarcode work ?

The main concept of this class is to generate the “labels” (barcode and text) and put them into a grid pattern. So we have two function:

• generate_label generate the single “label”, one of the time
• get_pdf generate the pdf putting the “labels” in a grid pattern

If you need, this class can be used without Django, in something like a tool for terminal  

Why do it?

One of the best arguments for having a personal ActivityPub site/server is you having a unique identity, don’t fragmenting your identity¹ on multiple Feedverse accounts.

But I implemented Webmention and stuff like Brid   for having the likes, comments and all the stuff I have in the ActivityPub about my posts. So why I need to do it?

If I need more reference to your post/article on the Feedverse, Mastodon has a function² for show your account under your link making a link to your profile. It also easy to set up and some platform has a plugin for it (like for Wordpress) or easy articole online for your favorite platfome (for example this is for Ghost   ).

But if you aren’t in the FeedVerse you are loosing in a way in to a social media moviment which is groing with major players studing and testing the ground (Meta, Tumblr, with also X/Twitter looking without public testing) and more and more time pass, more and more service integrate the ActivityPub in their social/site/service.

I also have a lot of attention from the tech blogger, where if you don’t write about excitement³ you are writing about how to implement ActivityPub on your site[^4].

And so? I need to integrate this new stuff into my site?

No and yes. If you want a presence in the FeedVerse you need to be active also there but no with a integration with the ActivityPub protocol: you can share it like the “old days”.

Also having a ActivityPub implementation somewhere is a resource draining server so you need to think about how much money and time you want to spend on it.

In the end Webmention and ActivityPub are two way to have more interaction with your site but you also need to now which interaction is better for your site.

The code

I wrote a ClassView for this case, this class. ¹

class CsvCarsDownload(View):
    def get(self, request, *args, **kwargs):
        response = HttpResponse(content_type="text/csv")
        response["Content-Disposition"] = 'attachment; filename="cars.csv"'
        my_dict = [{
          "brand": "Ford",
          "model": "Mustang",
          "year": 1964
        }]
        writer = csv.DictWriter(
            response, dialect="excel", fieldnames=my_dict[0].keys()
        )
        writer.writeheader()
        for element in my_dict:
          writer.writerow(element)
        return response

You can change the type of the the CSV changing the dialect with one of the type of the CSV dialect (excel, excel_tab, unix_dialect) and changing the my_dict with any type of list of dict.

With this code you can use any Class Mixin from others module for add other functions like permission supports or loggeer configurations.

“Imitation is the sincerest form of flattery that mediocrity can pay to greatness.”

Oscar Wilde

The basic idea come from a post by Brandon’s Journal   where Brandon response to an article and use the Re: prefix, like a email threads in the title of the post…

And I add a microformat following the IndieWeb stuff for the response tags. So my post which are a “Re:” will have a u-in-reply-to in the class of the links for implementing the IndieWeb, like in this article.

Blogroll

One of the best page I find on a blog. This is a page where the blogger show a link of site/blog (a html page or a Opml file) of interesting stuff.

I find iteresting because I can have an idea about the blogger interest and I can find new and different source of post.

Mine is here

Connect

Half of the old About Us page. In this page the blog put all the link needed to follow and/or contact the author on the classic way (email,…) and the new social way (mastodon, twitter, etc…)

Mine is here

Feeds

Inspired by this post   the feeds page is the most functional page for user on a site. This page has all the feeds’ links for the blog and other service the blogger is showing. Sometime some feeds aren’t from the blog but from other platform of the blogger.

Mine is here

Now

The other half of the About Us page. It come from an idea   about having a space (in this case a page on a site) about what are you doing with your life and interest. In this way you can also use it for showing off your little project or have a space for fundraising your project.

Sometime I search on Google now pages for find new project and others things.

Mine is here

Uses

The strange page (for me). There is a uses site   where you can send your Uses link where you share what tech you use every day or for the project. In this way you can check what tech other people use and for what.

In this type of pages I love to check what kind of coding and photo set up they have.

Mine is here

So?

So I suggest to have some of this pages on your blog if you thing of your blog as a personal blog. It can be usefull for start a conversation online or for having a place to send people for answer about your works and your stuffs.

IndieWeb the definition

I thing that, to explain IndieWeb we need two definition: one from a more “anthropological” point of view and one from a “tecnical/it” point of view

Anthropological definition

The #IndieWeb is for everyone, everyone who wants to be part of the world-wide-web of interconnected people. The social internet of people, a network of networks of people, connected peer-to-peer in human-scale groups, communities of locality and affinity.

Tecnical/It definition

The IndieWeb is a community of independent and personal websites connected by open standards, based on the principles of: owning your domain and using it as your primary online identity, publishing on your own site first (optionally elsewhere), and owning your content.

So, why I need to be in the Indieweb?

You don’t need to be in. You need to choose red pill or blue pill

Red Pill: If you want to have a look inside the mirror and, because you have something to say online, you need to have a personal domain and a space where you have the control and own your content and

You stay in Wonderland and I show you how deep the rabbit hole goes

Blue Pill: You keep the social network account you have and

The story ends, you wake up in your bed and believe whatever you want to believe

Red Pill, now what?

There are multiple things to do and tools to choose for you if you want to start with IndieWeb so I suggest a site which help you understand what you need to do and what to choose.

This site is Unplatform   from the people of the fromthesuperhighway .

This is a simple way to understand and start with the red pill and with a complete explanation for every tool you choose so you can have the easyest way to became an IndieWeb user.

Have a nice IndieWeb expirience

In the last X years¹ I understand the important of blogging and why a dev or a coder need to write a blog/newsletter/magazine about what is doing² and how big time sink is developing and maintaining a custom blog so I have this question:

For sharing Dev/Coding experiences is better Medium or StackOverflow?³

We need a “generic” Dev!

This is Bob. Bob is a BackEnd Developer, a Python Developer and he is learning some FrontEnd stuffs. So he wants to write about what he knows, what is learning and his doubts.

What a Dev need to share in a post

Bob need to share fragments of the code he writes, some screenshot (images) and a lot of texts. The latest are a simple request because it is the minimum requirements for all the blogging platform.

The code part is the “difficult” one.

The best situation is if the code is highlighted something like this

print("Hello World!")

and the platform need to support all the types of the files Bob’s code but we only need the highlight for the languages… If Bob needs to blog about Cobol or Pascal⁴

Customing and theming

Bob need to have a solid site/blog with a mobile and a desktop layout (more and more people read blog only with the tablet or the phone) and he need to make no changes or as little as possible for making HIS SITE/BLOG.

He also need a notification system for the followers (newsletters, feed, app with notification, …) and a SEO integration (analytics, meta, etc…) for becoming someone on the web.

Less edit he must do for posting better the platform is.

Money

Bob is a developer for money so he doesn’t want to spend money so… The all budget is 10€ (all needed for the domain) so the platform is a good one if it is free but is better if pay Bob.

So he needs a free platform⁵ and some way to make money from the posts (advertising, subscriptions, etc…)

And now the platforms

Now we know what Bob needs and wants, we find some platforms.

Wordpress, Jekill, Ghost, GoHugo and some other platforms aren’t the “right one” because they aren’t a “out of the box” expirience. You need to customized it and you need to have time for the updating of the platform soo we have SubStack and Medium as the platform.

Medium

The old one, Medium is the platform which is trying to become a blogging space for magazine and journal’ blogs. He is online from the 2012 and have all the need for a writer, with tool for automatic sharing, automatic posting and seo tools Medium is a complete platform.

SubStack

The new one, SubStack is the platform with two faces: one is a blogging space and the other is the mail-list space. SubStack out of the box way to work is a mail list for your followers, but the site part is not a bad one. It has automatic sharing, automatic posting and a micro-blogging part. But for SEO and promotion it all on you.

It also has one of the better subscribe platform of the web, where you can set some post to pay to read and some free to read.

Some other considerations

After all this considerations I need to show somethings behind the curtain : Medium has a lot of negative attention[^mediumproblem] because they have a lot of low quality post⁶ and SubStack has its share of problem with the paywall on the newsletters and some problems for the authors.⁷

In my opinion, if you want to blog about what are you doing, do it and this two platform can be a good starting point for start with your blogging journey but if it become something more you need to invest time in something more tailor to your need and understand what you need and what the platform give you.

If someone make me choose one between Medium and SubStack, I will choose SubStack because I love the idea of having a newsletters but I also feel the limitation of the close platform, bacause I had years and years of blogging on platform I hacked every way.

For example this blog is a GoHugo installation with a out of the box theme but I put so many changing in the code I have something more similar of the Monster of Frankenstein than a well coded theme…

One of them was a bigger ispiration for my implementation¹ and I have some chat with the author of the post over Mastodon for the implementation of the syndications link in a static site² and how to implement it.

Fundor333 @fundor333@mastodon.social

@chringel How do you fill the syndications' links? By hand?

10:56 AM · Aug 2, 2022

Christian 👼 @chringel@fosstodon.org

@fundor333 Yeah, unfortunately that‘s how I do it.

1. Publish a post on my blog
2. Post the link on Twitter/Mastodon/…
3. Update syndication param in front matter and republish

Haven‘t found another way yet.

6:15 PM · Aug 2, 2022

After 3 years I find the solution in a post about webmetions!

The input

Paul Kinlan wrote an article in the 2019 which flow under my radar and I didn’read it untile recently³.

In this post he use a Github action to write the webmention into a data folder for his GoHugo implementation. He has a script which download the data and put into file (the name of the file is the hash of the url of the post) which are use to render the webpage interested by the webmention. And all of this into Github Actions…

So I start thinking about it more and more… What can I have the syndications links into a file for url? Like the webmention in this post? I need to read the urls form the social media and other stuff.

But now with X/Twitter in bad shape and Facebook/Meta only good for share photos I only use Mastodon so it is easy read them and write a file for url…

A implementation I find is this one by Brandon Rozek⁴ that I like in a lot of way but I change it because I don’t use Medium…

The implementation

So I start with the development:

1. I need a file type/data structure standard for the Syndication
2. I need a script (or more) that find the syndication’s links
3. I need to edit/add to the template some code for the render of it

Structure data

First are the data structure: json, with the name of the file is an hash of the url

{
	"syndication": [
		"https://mastodon.social/@fundor333/113809667070052761"
	]
}

For example this is my data for this post . I choose Json because I can make check on it and it is a beautiful format for data

The Scraper script

In this case I find that Mastodon generate a feed for every public profile so I only implementa a reader (class MastodonFinder) which read the feed and save the toots url if find a link to my domain, and a writer class (WriterSyndication) which run all the other class (in this case only the Mastodon one) and write the output as a look-a-like of the json show before.

import feedparser
from pathlib import Path
import os
import json
import hashlib

domain = "https://fundor333.com"
rss_url_mastodon = "https://mastodon.social/@fundor333.rss"


def clean_slug(slug: str):
    return hashlib.md5(
        (slug.split("?")[0]).encode("utf-8"), usedforsecurity=False
    ).hexdigest()


class MastodonFinder:
    def find_urls(self, string):
        x = string.split('"')
        res = []
        for i in x:
            if i.startswith("https:") or i.startswith("http:"):
                res.append(i)
        return res

    def run(self, rss_url: str, domain: str, output: dict):
        feed = feedparser.parse(rss_url)
        if feed.status == 200:
            for entry in feed.entries:
                link = entry.get("link")
                for e in self.find_urls(entry.get("description")):
                    if domain in e:
                        e = clean_slug(e)
                        if output.get(e, False):
                            output[e].append(link)
                        else:
                            output[e] = [link.strip()]
        else:
            print("Failed to get RSS feed. Status code:", feed.status)


class WriterSyndication:
    def __init__(self, rss_url_mastodon: str, domain: str):
        self.output = {}
        self.rss_url_mastodon = rss_url_mastodon
        self.domain = domain

    def data_gathering(self):
        m = MastodonFinder()
        m.run(self.rss_url_mastodon, self.domain, self.output)

    def write(self):
        for key in self.output.keys():

            path_folder = os.path.join("data", "syndication")

            Path(path_folder).mkdir(parents=True, exist_ok=True)
            path_file = os.path.join(path_folder, key)

            with open(path_file + ".json", "w") as fp:
                json.dump({"syndication": self.output[key]}, fp)

    def run(self):
        self.data_gathering()
        self.write()


w = WriterSyndication(rss_url_mastodon, domain)
w.run()

I wrote in this way because I can change the readers or add new one but all the rest of the code remain the same.

TODO one bug

Only thing is I need to implement some code for reading all the old file and add to them and not create a new data file if there is an old one.

The GitHub Action

I run it as a Github Action with this parameters.

name: Syndication Link

on:
  schedule:
    - cron: "0 */2 * * *"
  workflow_dispatch:
jobs:
  webmentions:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@master

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.13"
          cache: "pip" # caching pip dependencies

      - name: Install Pip dependencies
        run: pip install -r requirements.txt

      - name: Fetch Syndication
        run: python ./action_script/syndication-collector.py

      - name: Commit to repository
        env:
          GITHUB_TOKEN: ${{ secrets.TOKEN }}
          COMMIT_MSG: |
            👾Fetch webmentions
            skip-checks: true
        run: |
          git config user.email "git@fundor333.com"
          git config user.name "fundor333"
          git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/fundor333/fundor333.github.io.git
          git checkout main
          git add .
          git diff --quiet && git diff --staged --quiet || (git commit -m "${COMMIT_MSG}"; git push origin main)

I always put “workflow_dispatch” as one of the running condiction for having a manual button for running it.

The template

This is the template fragment I use to show the syndication of one of the post. It return a single line of text with all the link of the syndication label as the host of the service (as if it is a mastodon.social/xxxx/xxxx link, the label will be mastodon.social).

{{ $urlized := .Page.Permalink | md5 }}
{{ if index .Site.Data.syndication $urlized }}

<hr>
<br>
  <div class="syndication">
      <i class="fas fa-link"></i>
      This post was also syndicated to:
      {{ $data:=  index .Site.Data.syndication $urlized  }}

      {{ $data:= $data.syndication }}

      {{ range $index, $url := $data}}
        {{- $parsed_url := urls.Parse $url -}}
        {{- if $index }}, {{- end }}
        <a class="u-syndication" href="{{ $url }}" rel="syndication">{{ $parsed_url.Host }}</a>
      {{ end }}
  </div>
  <br>
{{ end }}

I also add the microformat2 tags because all my site has them.

Conclusion

I love this way and I am happy because I resolve a problem I had with my blog for 3 years