Data wrangling is the process of cleaning and unifying messy and complex data sets for easy access and analysis. It can also mean fixing data that is somehow broken and unreadable.

Reparing a mp4 video clip

An example could be repairing a video file that was closed improperly. The reason could be:

• a an empty cell-phone battery causing a crash of the camera app, 
• a bad connection to an sdcard (i.e. physical contact issues),
• a media that is full or corrupted or,
• some other unspecified reasons.

Here we document a real example of an mp4 video clip which became corrupted due to the cellphone running out of power and the thus a premature end/crash of the android camera app.

In this case the error reported by VLC was:

(base) jan@beethoven:~/Desktop$ vlc 20211012_123750.mp4 
VLC media player 3.0.11.1 Vetinari (revision 3.0.11.1-0-g52483f3ca2)
[00005587c32405b0] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fca500ab3c0] moov atom not found
[00007fca18007de0] mp4 demux error: MP4 plugin discarded (no moov,foov,moof box)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fca18c15400] moov atom not found
[00007fca18007de0] avformat demux error: Could not open /home/jan/Desktop/20211012_123750.mp4: Unknown error 1094995529
qt.qpa.xcb: QXcbConnection: XCB error: 3 (BadWindow), sequence: 1590, resource id: 6299190, major code: 40 (TranslateCoords), minor code: 0
QObject::~QObject: Timers cannot be stopped from another thread

In this case the so called moov atom not found message is an indication of an improperly closed video file. Problems like this are not unusual and mostly easy to fix. There are many online tools available most of which are not much more than wrappers around the famous ffmpeg command line tool. 

Sometimes a simple command like the one below can already be enough to fix the video clip. Sometimes it is a bit more involved. The easiest way to fix this is to download and compile from github the untrunc tool that uses avcodec libraries to effectively repair the mp4 video clip.

ffmpeg -i 20211012_123750.mp4 -c copy fixed_output.mp4

This did not work and produced an error message similar to the one mentioned above hence we proceed with the untrunc tool as follows.

sudo apt-get install libavformat-dev libavcodec-dev libavutil-dev
mkdir ~/source
cd ~/source
git clone https://github.com/anthwlock/untrunc.git
cd untrunc
make
./untrunc ~/Desktop/good.mp4 ~/Desktop/20211012_123750.mp4 

A sample file named #good.mp4 recorded on the same device is supplied to extract correct metadata from. This should produce a fixed version of the input file with the _fixed.mp4 suffix. All that remains now it to test it with the vlc player.

(base) jan@beethoven:~/source/untrunc$ vlc ~/Desktop/20211012_123750.mp4_fixed.mp4