Remote Log File Monitoring with Bash and Netcat

Yeah, you can probably just shell right up in there.

ssh root@obscure-server.com tail -f /var/log/syslog

But sometimes you just need those logs to come straight to you. Maybe to jump onto a different network.

We still have to shell in briefly to kick it off of course, but that only takes a moment.

I present to you, a script named fu (file over udp).

#!/bin/bash
LAST=$(tail -n 1 $1)
while [ 1 ]
do
    sleep 0.0001
    CUR=$(tail -n 1 $1)
    if [ "$LAST" != "$CUR" ]; then
        LAST=$CUR
        printf '%s\n' "$LAST" > "/dev/udp/$2/$3"
    fi
done

Dumb, insecure, and full of ignored edge cases, but it works just well enough to not warrant another ounce of effort.

We can shell in, spawn it, and leave:

nohup ./fu /var/log/syslog obscure-client.com 12345 &

And presto, obscure-server.com will start forwarding new lines appended to /var/log/syslog to port 12345 of  obscure-client.com until death.

Then on obscure-client.com, we can use netcat to print out the new log lines as they as added. 

nc -lukvw 0 12345

Of course, it's way more useful to simply spawn a persistent reverse SSH tunnel instead of fu instance.

ssh-copy-id root@obscure-client.com
nohup ssh -N -R 8000:localhost:22 root@obscure-client.com &

Then you can ssh from obscure-client.com to obscure-server.com whenever you want, regardless of those pesky firewalls.

ssh localhost -p 8000 

Seems like stuff every hacker knows by heart, but I am no hacker.

Prevent FFMPEG from removing KLV data stream from MPEG-TS output

ffmpeg is a great tool for recording live video streams to files, clipping video files, streaming video files to udp, and a bunch of stuff I don't care about. Of course, I feel like it's always a battle to get the right flags to do exactly what I need.

The most recent example, I just want to copy an MPEG-TS file:

ffmpeg -i ./CheyenneVAhospital.ts out.ts

Yeah, of course it isn't that easy. ffmpeg has decided that I have no need for the KLV data stream.

Input #0, mpegts, from './CheyenneVAhospital.ts':

  Duration: 00:01:05.15, start: 271.125322, bitrate: 5535 kb/s

  Program 1 

  Stream #0:0[0x100]: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 127 kb/s

  Stream #0:2[0x102]: Data: klv (KLVA / 0x41564C4B)

Stream mapping:

  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (native))

  Stream #0:1 -> #0:1 (aac (native) -> mp2 (native))

Press [q] to stop, [?] for help

Spoiler: I do need it. Very much. And after trying many flags (codec:data, map_metadata, copy_unknown, etc.), I found that I had to explicitly map all three stream types.

ffmpeg -i ./CheyenneVAhospital.ts -map 0:v -map 0:a -map 0:d out.ts

Input #0, mpegts, from './CheyenneVAhospital.ts':

  Duration: 00:01:05.15, start: 271.125322, bitrate: 5535 kb/s

  Program 1 

  Stream #0:0[0x100]: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 127 kb/s

  Stream #0:2[0x102]: Data: klv (KLVA / 0x41564C4B)

File 'f.ts' already exists. Overwrite? [y/N] y

Stream mapping:

  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (native))

  Stream #0:1 -> #0:1 (aac (native) -> mp2 (native))

  Stream #0:2 -> #0:2 (copy)

Press [q] to stop, [?] for help

Simple enough, but I am compelled by future me to preserve this knowledge in the cloud so that I can further delay defragmenting my brain.

Using multicat to send an MPEG-TS video file to a UDP multicast address

Multicat is fairly simple to use. Just give it an input and output and it figures the rest out on it's own. 

multicat -uU ./CheyenneVAhospital.ts 226.6.6.1:8209

The -uU flags to ignore the missing RTP headers.

But wait! You need a .aux file apparently

error: couldn't open file ./CheyenneVAhospital.aux (No such file or directory)

Piece of cake. The docs say you just have to ingest the file with the ingests command. Let's copy and paste that into our shell

ingests -p 68 ./CheyenneVAhospital.ts

Oh great. That didn't work.

debug: end of file reached

error: no PCR found

Alright our PCR PID must be wrong. Luckily we can use ffprobe to get the PCR PID out of a .ts file.

ffprobe CheyenneVAhospital.ts -show_programs | grep pcr_pid

...

pcr_pid=256

Oh lovely! Why can't ingests do that? I dunno man stop asking questions.

Let's try ingests with the correct PCR PID

ingests -p 256 ./CheyenneVAhospital.ts

Hey, cool. No errors this time. And our .aux file is more than 0 bytes!

Now we can finally do the thing.

multicat -uU ./CheyenneVAhospital.ts 226.6.6.1:8209

Oh by the way, if you want to change your MPEG-TS file to have a consistent video bitrate, you can use ffmpeg for that. Here's an example of converting the video stream to 1Mb/s

ffmpeg -i ./CheyenneVAhospital.ts -c:v libx264 -b:v 1M -map 0 CheyenneVAhospital_1M.ts

And multicat doesn't support looping, so you'll just have to put the multicat command inside a loop body and deal with the spikes

for (( ; ; ))

do

multicat -uU ./CheyenneVAhospital_1M.ts 226.6.6.1:8209

done

VBoxManage: error: The virtual machine 'minikube' has terminated unexpectedly

Full error: 

Exiting due to PR_VBOX_BLOCKED: Failed to start host: creating host: create: creating: Unable to start the VM: /usr/bin/VBoxManage startvm minikube --type headless failed:

VBoxManage: error: The virtual machine 'minikube' has terminated unexpectedly during startup with exit code 1 (0x1)

VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine

Solution:

# Purge VirtualBox.

sudo killall VirtualBox

sudo apt-get remove --purge virtualbox

rm -rf ~/.config/VirtualBox/

# Wipe minikube cache

rm -rf ~/.minikube/

# Reinstall VirtualBox

sudo apt-get install virtualbox

# Rebuild minikube VM

minikube start --driver=virtualbox

Solution in one line:

sudo killall VirtualBox; rm -rf ~/.config/VirtualBox/; rm -rf ~/.minikube/; sudo apt-get remove -y --purge virtualbox && sudo apt-get install  -y virtualbox && minikube start --driver=virtualbox

Convert PostGIS ST_LineString to ST_MultiPoint

Sometimes you need to convert a PostGIS line into a multipoint geometry. I mean, how else did you find this?

SELECT ST_Union(

    ARRAY(

        SELECT (ST_DumpPoints(linestringcolumn)).geom

        FROM atable

    )

);

Just kidding, use ST_Points(). Why ST_MultiPoint() does not do this is a mystery to my tiny brain. I would ask Dan Baston, but his answer would probably fracture my mental model of reality, and I don't have time to reconstruct it right now.

SELECT ST_Points(linestringcolumn) FROM atable;

Fix for npm ERR! command-line line 0: "unsupported option accept-new"

I encountered this on a server with a newer npm version and an older openSSH version. A few node dependencies were stored in private git repositories, and npm just simply wouldn't fetch them.

The fix was to downgrade npm to v6

    npm i -g npm@6

Unreal Engine 4 pointer cursor stuck on web browser widget

I know UE4's web browser widget is an "experimental" feature, but this is a pretty obnoxious bug that they have refused to fix for at least 4 years.

As far as I can tell, as soon as your cursor enters the "pointer" state while inside the browser widget, it will be stuck there indefinitely, even after you close the widget. The only solution I have found is to prevent it from ever reaching that state.

If you have full control over the webpage you are displaying, you can modify your CSS to use some other cursor type for all your elements. I did not.

My solution? Set the cursor for every element to 'default' using javascript.

var nodes = document.querySelectorAll('*');

for(var i=0; i<nodes.length; i++) {

    nodes[i].setAttribute('style', 'cursor:default !important'); 

}

When? All of the time.

It does seem excessive to do every element every tick, and it probably is, but when you have no control over a webpage, you never know when some new pesky element is going to be shoved into the DOM and screw you again. Why risk it?

One thing I did try was to set a software "Hand" cursor. I assume this translates to cursor:pointer, but who knows? That did nothing at all. Maybe the bug here is that there is no "pointer" cursor type in UE4, so the web browser widget has no idea how to turn it off.

One thing I did NOT try was to set up invisible hardware cursor images. Perhaps there is a conflict in the transition from hardware to software cursors? I do not know, but I do know that I end up with both when I mouse out of an HTML button. Forever.