IPv6 Multicast Listener for Elixir

I have yet to find a working example of this, so I am obligated to post one. Especially considering the frustration I and a colleague more intelligent than I had to undergo before we found a working solution.

Please note that you will not get away with using an old version of OTP. IPv6 Multicast joins were broken for a long time.

Here is a simple script that will listen for a single packet, print it out, and exit.

require Logger
address={0xff1e, 0, 0, 0, 0, 0, 0x70, 0x25}
iface=0
{:ok, socket} = :gen_udp.open(8208, [
{:ip, address},
{:add_membership, {address, iface}},
{:debug, true},
:binary,
{:reuseaddr, true},
{:recbuf, 8388608},
])
Logger.info("UDP socket opened successfully #{inspect(socket)}")
{:ok, message} = :gen_udp.recv(socket, 9)
Logger.info("Received message: #{inspect(message)}")

An interface index of 0 works for global multicast addresses. Your kernel should figure it out based on the routing table. For a link-local or unique-local address, you will need to set that to your actual interface index.

NOTE THE ORDER OF THE OPTIONS

If you move :add_membership lower down, you will get a nonsensical eaddrinuse error. Yeah, that took a while to figure out. The other options can be moved around, but add_membership needs to be up towards the top. We assume this is because the options are processed in the order in which they are provided instead of the order of which they should be.

So let's move on to the gen_server implementation to expose the next gotcha.

defmodule Listen do
use GenServer
require Logger
@address {0xff1e, 0, 0, 0, 0, 0, 0x70, 0x25}
@iface 0
#@address {224,0,0,1}
#@iface {0,0,0,0}

def start_link(_) do
GenServer.start_link(__MODULE__, {}, name: __MODULE__)
end

def init(_opts) do
{:ok, socket} = :gen_udp.open(8208, [
{:ip, @address},
{:add_membership, {@address, @iface}},
{:debug, true},
:binary,
{:reuseaddr, true},
{:recbuf, 8388608},
{:active, true}
])
Logger.info("UDP socket opened successfully #{inspect(socket)}")
{:ok, %{socket: socket}}
end
def handle_info(msg, state) do
Logger.info("Received message: #{inspect(msg)} #{inspect(state)}")
{:noreply, state}
end
end
Listen.start_link([])
Process.sleep(:infinity)

Here, you must add {:active, true}, or handle_info() will never be called. Elixir will just leave the packets in the network queue. This is not true for IPv4. I left in the commented out IPv4 options if you want to prove this to yourself.

I would be remiss if I didn't provide you a command to test these.

echo "Elixir sucks" > /dev/udp/ff1e::70:25/8208

Before I leave, I will urge you to hold off on using elixir for important things. It is an immature pet project with poor performance, and you will be pressed to find engineers well versed in it who can maintain it with any efficiency.

Finding the maximum time_t that gmtime() will handle.

This is not another dry technical post. This problem is trivial to solve by even the most junior programmer. This post is to reflect on the current state of LLMs.

Being a trivial problem, I gave it to ChatGPT because I was too lazy to type it out myself, and it was not going into a production system. I just wanted to know the answer. It gave me this code, which made me laugh out loud:

#include <stdio.h>
#include <time.h>
#include <limits.h>

int main() {
    time_t t = 0;
    struct tm *timeinfo;

    // Test from 0 to the maximum value of time_t
    while (1) {
        timeinfo = gmtime(&t);
        if (timeinfo == NULL) {
            printf("gmtime() failed for time_t = %lld\n", (long long)t);
            break;
        }
        t++;
    }

    printf("Maximum time_t for gmtime(): %lld\n", (long long)(t - 1));
    return 0;
}


This code is technically correct, but I knew immediately that me and my machine would be dead by the time this function returned. So I took it and made it return instantly with a binary search:

#include <stdio.h>
#include <time.h>
#include <limits.h>

int main() {
    time_t t = 0, l = 0, u = LLONG_MAX;
    struct tm *timeinfo;

    while (1) {
        if (u - l < 2) {
             break;
        }
        t = (u+l) / 2;
        timeinfo = gmtime(&t);
        if (timeinfo) {
            l = t;
        } else {
            u = t;
        }
    }

    printf("Maximum time_t for gmtime(): %lld\n", (long long)(t));
}

I got my answer. I went back to ChatGPT and asked how long it would take to run the code it gave me on the world's fastest processor. It estimated 292,278,994 years.

But then something funny happened. I asked it to rewrite it's own code to execute in less than 1 second. It spit out a function almost identical to mine, only it was easier to read:

#include <stdio.h>
#include <time.h>
#include <stdint.h>

int main() {
    time_t low = 0;
    time_t high = INT64_MAX;  // Use the maximum possible time_t value (64-bit)
    time_t mid;
    struct tm *timeinfo;

    // Perform binary search to find the max time_t for which gmtime() works
    while (low < high) {
        mid = low + (high - low) / 2;  // Calculate the midpoint

        timeinfo = gmtime(&mid);  // Try to get the time in UTC

        if (timeinfo == NULL) {
            // gmtime failed, so the maximum valid time_t must be less than mid
            high = mid - 1;
        } else {
            // gmtime succeeded, so the maximum valid time_t must be greater than or equal to mid
            low = mid + 1;
        }
    }

    // At the end of the binary search, `high` will hold the maximum valid time_t
    printf("Maximum time_t for gmtime(): %lld\n", (long long)high);
    return 0;
}

 

That's kind of neat. Maybe it will take my job after all. Maybe I don't need coding skills. Maybe I just need to become a "Prompt Engineer".

But seriously, I now believe LLMs have savant syndrome, like the Rain Man if he was high on LSD. Not in the sense that they have amazing abilities or talent, but in the sense that they have various developmental disorders and no common sense.

So far, I have enjoyed coding alongside ChatGPT. Not because it makes me more productive, but it gives me that occasional comedy relief I need to make my job bearable. Working from home, I no longer get to clown around with QA on smoke breaks.

Now I suppose if you have the patience to ask ChatGPT enough of the right questions, you may eventually get a reasonable code snippet, but that may require you to already know enough of the answer to call out its BS. If you trust anything that comes out of this thing, you will be in for a ride.

I have gotten some mileage out of it though. It has been great as a learning tool. It saves me wading through search results to find simple answers. It rarely has the right fixes for the error messages I feed it, but it often points me in a good direction. If I know nothing about something, ChatGPT has consistently been a great place to start. It gives me all the truths and lies that I used to get from wikipedia, but in a faster and more entertaining way.

Remote Log File Monitoring with Bash and Netcat

Yeah, you can probably just shell right up in there.

ssh root@obscure-server.com tail -f /var/log/syslog

But sometimes you just need those logs to come straight to you. Maybe to jump onto a different network.

We still have to shell in briefly to kick it off of course, but that only takes a moment.

I present to you, a script named fu (file over udp).

#!/bin/bash
LAST=$(tail -n 1 $1)
while [ 1 ]
do
sleep 0.0001
CUR=$(tail -n 1 $1)
if [ "$LAST" != "$CUR" ]; then
LAST=$CUR
printf '%s\n' "$LAST" > "/dev/udp/$2/$3"
fi
done

Dumb, insecure, and full of ignored edge cases, but it works just well enough to not warrant another ounce of effort.

We can shell in, spawn it, and leave:

nohup ./fu /var/log/syslog obscure-client.com 12345 &

And presto, obscure-server.com will start forwarding new lines appended to /var/log/syslog to port 12345 of  obscure-client.com until death.

Then on obscure-client.com, we can use netcat to print out the new log lines as they as added. 

nc -lukvw 0 12345

Of course, it's way more useful to simply spawn a persistent reverse SSH tunnel instead of fu instance.

ssh-copy-id root@obscure-client.com
nohup ssh -N -R 8000:localhost:22 root@obscure-client.com &

Then you can ssh from obscure-client.com to obscure-server.com whenever you want, regardless of those pesky firewalls.

ssh localhost -p 8000 
Seems like stuff every hacker knows by heart, but I am no hacker.

Prevent FFMPEG from removing KLV data stream from MPEG-TS output

ffmpeg is a great tool for recording live video streams to files, clipping video files, streaming video files to udp, and a bunch of stuff I don't care about. Of course, I feel like it's always a battle to get the right flags to do exactly what I need.

The most recent example, I just want to copy an MPEG-TS file:

ffmpeg -i ./CheyenneVAhospital.ts out.ts

Yeah, of course it isn't that easy. ffmpeg has decided that I have no need for the KLV data stream.

Input #0, mpegts, from './CheyenneVAhospital.ts':

  Duration: 00:01:05.15, start: 271.125322, bitrate: 5535 kb/s

  Program 1 

  Stream #0:0[0x100]: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 127 kb/s

  Stream #0:2[0x102]: Data: klv (KLVA / 0x41564C4B)

Stream mapping:

  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (native))

  Stream #0:1 -> #0:1 (aac (native) -> mp2 (native))

Press [q] to stop, [?] for help

Spoiler: I do need it. Very much. And after trying many flags (codec:data, map_metadata, copy_unknown, etc.), I found that I had to explicitly map all three stream types.

ffmpeg -i ./CheyenneVAhospital.ts -map 0:v -map 0:a -map 0:d out.ts

Input #0, mpegts, from './CheyenneVAhospital.ts':
  Duration: 00:01:05.15, start: 271.125322, bitrate: 5535 kb/s
  Program 1 
  Stream #0:0[0x100]: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 127 kb/s
  Stream #0:2[0x102]: Data: klv (KLVA / 0x41564C4B)
File 'f.ts' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (native))
  Stream #0:1 -> #0:1 (aac (native) -> mp2 (native))
  Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help

Simple enough, but I am compelled by future me to preserve this knowledge in the cloud so that I can further delay defragmenting my brain.

Using multicat to send an MPEG-TS video file to a UDP multicast address

Multicat is fairly simple to use. Just give it an input and output and it figures the rest out on it's own. 

multicat -uU ./CheyenneVAhospital.ts 226.6.6.1:8209

The -uU flags to ignore the missing RTP headers.

But wait! You need a .aux file apparently

error: couldn't open file ./CheyenneVAhospital.aux (No such file or directory)

Piece of cake. The docs say you just have to ingest the file with the ingests command. Let's copy and paste that into our shell

ingests -p 68 ./CheyenneVAhospital.ts

Oh great. That didn't work.

debug: end of file reached

error: no PCR found

Alright our PCR PID must be wrong. Luckily we can use ffprobe to get the PCR PID out of a .ts file.

ffprobe CheyenneVAhospital.ts -show_programs | grep pcr_pid

...

pcr_pid=256

Oh lovely! Why can't ingests do that? I dunno man stop asking questions.

Let's try ingests with the correct PCR PID

ingests -p 256 ./CheyenneVAhospital.ts

Hey, cool. No errors this time. And our .aux file is more than 0 bytes!

Now we can finally do the thing.

multicat -uU ./CheyenneVAhospital.ts 226.6.6.1:8209

Oh by the way, if you want to change your MPEG-TS file to have a consistent video bitrate, you can use ffmpeg for that. Here's an example of converting the video stream to 1Mb/s

ffmpeg -i ./CheyenneVAhospital.ts -c:v libx264 -b:v 1M -map 0 CheyenneVAhospital_1M.ts

And multicat doesn't support looping, so you'll just have to put the multicat command inside a loop body and deal with the spikes

for (( ; ; ))

do

multicat -uU ./CheyenneVAhospital_1M.ts 226.6.6.1:8209

done

VBoxManage: error: The virtual machine 'minikube' has terminated unexpectedly

Full error: 

Exiting due to PR_VBOX_BLOCKED: Failed to start host: creating host: create: creating: Unable to start the VM: /usr/bin/VBoxManage startvm minikube --type headless failed:

VBoxManage: error: The virtual machine 'minikube' has terminated unexpectedly during startup with exit code 1 (0x1)

VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine


Solution:

# Purge VirtualBox.

sudo killall VirtualBox

sudo apt-get remove --purge virtualbox

rm -rf ~/.config/VirtualBox/


# Wipe minikube cache

rm -rf ~/.minikube/


# Reinstall VirtualBox

sudo apt-get install virtualbox


# Rebuild minikube VM

minikube start --driver=virtualbox


Solution in one line:

sudo killall VirtualBox; rm -rf ~/.config/VirtualBox/; rm -rf ~/.minikube/; sudo apt-get remove -y --purge virtualbox && sudo apt-get install  -y virtualbox && minikube start --driver=virtualbox



Convert PostGIS ST_LineString to ST_MultiPoint

Sometimes you need to convert a PostGIS line into a multipoint geometry. I mean, how else did you find this?

SELECT ST_Union(

    ARRAY(

        SELECT (ST_DumpPoints(linestringcolumn)).geom

        FROM atable

    )

);

Just kidding, use ST_Points(). Why ST_MultiPoint() does not do this is a mystery to my tiny brain. I would ask Dan Baston, but his answer would probably fracture my mental model of reality, and I don't have time to reconstruct it right now.

SELECT ST_Points(linestringcolumn) FROM atable;