Converting recorded wav calls to mp3

Just want to thank you all. You are my heroes. I was able to take a customer with 80 gb (3 months) of recordings and shrink it down to 16 gb. This solves our disk problems for past and future calls.

Thanks again!

2 Likes

I know this is an old post but I had to submit my 2 cents. First of all jhayes and dicko, I can’t thank either of you enough for your knowledge and direction. Using your scripts I was able to convert my client’s database of 2.7 million recordings to a reasonable size, and implement a long term storage strategy. You guys saved me a ton of time, and I can’t thank you guys enough. If you’re ever in Phoenix send me a message and the beers are on me!

Thanks guys,

Greg

2 Likes

Since this is the #1 Google result for this topic, I’m posting this here

I used the ideas from the scripts posted here to write a cleaner script with some improvements. Enjoy!

Notes:

  • LAME is automatically installed on the latest FreePBX distros, otherwise you can compile and install manually
  • You can change the LAME encoding parameters to better suit your needs
  • You need to specify the MySQL root password (or whatever user you like). Root password is blank by default (BTW, change it!)

EDIT: thanks to dicko for some improvements!

[code]#!/bin/bash

Save script as /var/lib/asterisk/bin/wav2mp3.sh and then:

chown asterisk:asterisk /var/lib/asterisk/bin/wav2mp3.sh

chmod +x /var/lib/asterisk/bin/wav2mp3.sh

Configure FreePBX postrecording script as: /var/lib/asterisk/bin/wav2mp3.sh ^{CALLFILENAME} ^{UNIQUEID}

Wait 3 seconds before we start

sleep 3

The default path for the recordings includes the current date

dy=$(date ‘+%Y’)
dm=$(date ‘+%m’)
dd=$(date ‘+%d’)

Path where recordings are stored, including trailing slash

dtpath=/var/spool/asterisk/monitor/$dy/$dm/$dd/

If the WAV file is 44 bytes long, delete it and exit. These are empty audio files that are generated

for example when a ring group or queue rings multiple extensions. This has been reported multiple

times, most recent bug report is https://issues.freepbx.org/browse/FREEPBX-13370

if [ $(wc -c < “$dtpath$1.wav” | cut -d ’ ’ -f1) -eq 44 ]; then
rm -f "$dtpath$1.wav"
exit 0
fi

Use LAME for the MP3 conversion, on a low priority process

nice lame -b 32 -m m “$dtpath$1.wav” “$dtpath$1.mp3”

If the conversion was successful, update CDR database and delete WAV file

if [ $? -eq 0 ]; then
mysql -u root -p’rootpassword’ -s -D asteriskcdrdb -e "UPDATE cdr SET recordingfile = ‘$1.mp3’ WHERE uniqueid = ‘$2’"
rm -f "$dtpath$1.wav"
else
exit 1
fi[/code]

1 Like

A few points, your $dtpath is actually available as ${MIXMON_DIR}, the script is run as asterisk so you don’t need the chown, You don’t need to modify the $PATH, all the binaries called are already available to the asterisk user, you won’t have a problem with 44 byte files as lame will fail and your deleting of the *.wav file will take care of that you should paramatize ‘rootpassword’ at the top of the script “rootpassword=theoneyouuse” if you have one, Good idea to update the asteriskcdrdb though.

1 Like

Thanks for the corrections!

Regarding the {MIXMON_DIR} variable, I was never able to pass it properly from the post recording script of FreePBX, my script receives an empty string (I don’t know why). Where you able to successfully use it?

And regarding the 44 bytes files, I still think it’s needed because LAME does not fail, it outputs an 864 bytes file, which after some months of heavy usage can add up to several GB.

Also, this method of updating the asteriskcdrdb is much more efficient because the uniqueid field is indexed :wink:

Best regards!

1 Like

you can just echo $n or whatever where $n is ${MIXMON_DIR} if you doubt it just try it, never had a problem with that, I personally use sox with liblame compiled in as it is quicker, soxi -D will give you the real length of the file, I don’t disagree with the mysql bit though as I stated.

1 Like

First of all a big THANK YOU dicko, georgeman and jhayes for all your work.

Just wanted to add my 2 cents. For all the cautious kind like me, i made some small changes so that all the WAV files are moved untill you’re confident you can start deleting them.

Also removed the password for novice users like me so that they can simply copy paste and start using it. most ‘standard’ freepbx installations won’t have a MySQL password.

NOTE: In order to enable “Post Call Recording Script” in freepbx goto SETTINGS > ADVANCED SETTINGS
Change both “Display Readonly Settings” & “Override Readonly Settings” to YES

#!/bin/bash
# ---
# Save script as /var/lib/asterisk/bin/wav2mp3.sh and then:
# chown asterisk:asterisk /var/lib/asterisk/bin/wav2mp3.sh
# chmod +x /var/lib/asterisk/bin/wav2mp3.sh
# ---
# Configure FreePBX postrecording script as: /var/lib/asterisk/bin/wav2mp3.sh ^{CALLFILENAME} ^{UNIQUEID}
# ---

# Wait 7 seconds before we start
sleep 7

# The default path for the recordings includes the current date
dy=$(date '+%Y')
dm=$(date '+%m')
dd=$(date '+%d')

# Path where recordings are stored, including trailing slash
dtpath=/var/spool/asterisk/monitor/$dy/$dm/$dd/

# Path where WAV are moved, including trailing slash
wavpath=/var/spool/asterisk/monitor/wav/$dy/$dm/$dd/

# If the WAV file is 44 bytes long, delete it and exit. These are empty audio files that are generated
# for example when a ring group or queue rings multiple extensions. This has been reported multiple
# times, most recent bug report is https://issues.freepbx.org/browse/FREEPBX-13370
if [ $(wc -c < "$dtpath$1.wav" | cut -d ' ' -f1) -eq 44 ]; then
	rm -f "$dtpath$1.wav"
	exit 0
fi

# Use LAME for the MP3 conversion, on a low priority process
nice lame -b 32 -m m "$dtpath$1.wav" "$dtpath$1.mp3"

# If the conversion was successful, update CDR database and delete WAV file
if [ $? -eq 0 ]; then
	mysql -u root -s -D asteriskcdrdb -e "UPDATE cdr SET recordingfile = '$1.mp3' WHERE uniqueid = '$2'"

# Move to a WAV folder instead of deleting the files 	
	mkdir -p "$wavpath"
	mv "$dtpath$1.wav" "$wavpath$1.wav"
	
# Uncomment	when you are confident and want to start deleting the files, comment the section above to stop moving the files.
# rm -f "$dtpath$1.wav"

else
	exit 1
fi

I have been convertine wav’s to MP3’s for years and saving to an external storage server using a fuse mounted directory, under the “monitor” directory.
Recently I switched to a new-build FreePBX14 (official distro), which I love by the way, but now my scripts are buggy and I am hoping that someone in here will see something that I am missing, and help be get it working correctly…

The problem isn’t huge, but a real PITA because I have to remember to maually execute the job late in the day, or it won’t finish before midnight, and then I have un-converted files.

The Problem is simply:
When step 1 (MP3convet.sh) is executed by crontab, it processes ONE file instead of running all available.
If I manually execute /usr/local/bin/MP3convert, it will run everything available at the time I execute it.

The whole process takes about 12 seconds including logging into the db, loggingq and reporting

Basically I have a .sh that starts the main script, then reports to a couple email addresses:

Step 1 (crontab): */05 * * * * /usr/local/bin/wav2mp3/MP3convert.sh (FYI - I tried 10 minutes, 5 minutes, and 1 minute)

Step2: MP3convert.sh

#!/bin/bash

TODAY=date +%F
echo “
–START-------------
date >> /tmp/MP3convert.log
/usr/local/bin/wav2mp3/ls_rec_file.pl $TODAY | tee -a /tmp/MP3convert$TODAY.log | mail -r [email protected] -s “CES MP3 Convert” [email protected]

Step 3: The mack-daddy which handles time stamp, remove silence, convert to mp3, and update CDR to reflect wav to mp3 extension. (note that I added some dots [ ie .#] to stop the post from having bold letters)

#!/usr/bin/perl -w

.# Script to post process recordings in Asterisk / FreePBX

.# You can set in FreePBX the “Run After Record” parameter
.# to be:

.# /var/lib/asterisk/bin/record_runafter.pl ^{UNIQUEID} ^{MIXMONITOR_FILENAME}

$|++;
use strict;
use DBI;
my %config;
my $dbh;
use File::Basename;
use File::Copy;

my $DB = $ENV{‘DEBUG’};

#Get the date from the environment OR ARG[0]
my $QDate = $ENV{‘QDATE’} || $ARGV[0];
die “$0: QDATE not set nor ARG1 set.” unless $QDate;

.# Destination Directory for recordings
$config{‘destDir’} = “/var/spool/asterisk/monitor/”;

.# What app do we use to encode?
$config{‘LAME’} = which lame;
chomp($config{‘LAME’});
die "$0: Lame not found " unless $config{‘LAME’};

#What app do we use to remove silence
$config{‘RMSILENCE’} = ‘/usr/local/bin/wav2mp3/rmsilenceWAV.sh’;
die “$0: $config{‘RMSILENCE’} not found.” unless -x $config{‘RMSILENCE’};

.# Read FreePBX configuration
open( CONFIG, “</etc/amportal.conf” ) or die(“Could not open /etc/amportal.conf. Aborting…”);
while () {
chomp;
$_ =~ s/^\s+//g;
$_ =~ s/([^#]*)#/$1/g;
$_ =~ s/\s+$//g;
if ($_ ne “”) {
my($key,$val) = split(/=/);
$config{$key}=$val;
}
}

$config{‘cdrHost’} = $config{‘CDRDBHOST’} ?$config{‘CDRDBHOST’}:$config{‘AMPDBHOST’};
$config{‘cdrDBName’} = ‘asteriskcdrdb’;
$config{‘cdrTableName’} = $config{‘CDRDBTABLENAME’} ?$config{‘CDRDBTABLENAME’}:‘cdr’;
$config{‘cdrUser’} = $config{‘CDRDBUSER’} ?$config{‘CDRDBUSER’}:$config{‘AMPDBUSER’};
$config{‘cdrPass’} = $config{‘CDRDBPASS’} ?$config{‘CDRDBPASS’}:$config{‘AMPDBPASS’};
$config{‘monitor_base’} = $config{‘MIXMON_DIR’} ? $config{‘MIXMON_DIR’} : $config{‘ASTSPOOLDIR’} . ‘/monitor’;

#my $uniqueid = $ARGV[0];
#my $directorio_archivo = $ARGV[1];
#my $type = $ARGV[2];

print “\n$=$config{$}” foreach qw/ cdrHost cdrDBName cdrTableName cdrUser cdrPass AMPDBHOST /;

sub connect_db() {
my $return = 0;
my %attr = (
PrintError => 0,
RaiseError => 0,
);
my $dsn = “DBI:mysql:database=$config{‘cdrDBName’};host=$config{‘AMPDBHOST’}”;
$dbh->disconnect if $dbh;
$dbh = DBI->connect( $dsn, $config{‘cdrUser’}, $config{‘cdrPass’}, %attr ) or $return = 1;
return $return;
}

my $time = localtime(time);
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localtime(time);
$mon++;
$year += 1900;

#Change upper time limit to 2 minutes back
my $Today = sprintf ("%4d-%02d-%02d",$year,$mon,$mday);
if ( $min < 2 ){
$hour–;
$min = ‘58’;
} else {
$min–;
}

my $Date2 = $Today . " " . sprintf("%02d:%02d:%02d",$hour,$min,‘0’) ;

# Connect to Database

&connect_db();

print “\nTODAY=$Today QDate=$QDate through $Date2”;

print “\nStarting …”;
my $query = “SELECT calldate,recordingfile,duration,cnam,lastapp
FROM $config{‘cdrTableName’}
WHERE DATE(calldate) = ‘$QDate’ AND calldate < ‘$Date2’ AND recordingfile > ’ ’ AND SUBSTRING(recordingfile, -4) = '.wav’
ORDER BY calldate
limit 4000”;

my $sth = $dbh->prepare($query) or die “SQL-do Error: $DBI::errstr\n”;
$sth->execute or die “SQL Error: $DBI::errstr\n”;

#Let recordings settle before we start if we are processing Today
sleep 10 if $Today eq $QDate;

my $NumRec = 0;
while( my $info = $sth->fetchrow_hashref){
$NumRec++;
my %i2 = %{$info}; #Easier to read
my @subdir = split( ‘-’, substr($i2{‘calldate’},0,10));
my $file = $config{‘monitor_base’} . ‘/’ . join (’/’ , @subdir) . “/$i2{‘recordingfile’}”;
$config{‘archiveDir’} = “/var/spool/asterisk/wavarchive/” . join (’/’ , @subdir) . ‘/’;
my ($archivo, $directorio, $suffix) = fileparse($file , ‘.wav’ );
printf “\n%5.0d %-60s " , $NumRec,”$archivo$suffix" ;#archive=$archivo directory=$directorio suffix=$suffix ";
next unless $suffix eq ‘.wav’;
#print " archive=$archivo directory=$directorio suffix=$suffix “;
if ( -f $file ){
mp3me($dbh,$file,$i2{‘recordingfile’},$i2{‘calldate’},$i2{‘cnam’},$i2{‘lastapp’});
print " Updated to MP3”;
} else {
print “Not Found! Duration $i2{‘duration’}”;
my $NewFile = $file;
$NewFile =~ s/.wav$/.mp3/;
if ( -f $NewFile ){
print “\n$file\n$NewFile\n OH an mp3 exists need to update the database record\n”;
my $Mquery = “
UPDATE cdr
SET recordingfile = REPLACE(recordingfile, ‘.wav’, ‘.mp3’)
WHERE calldate = ‘$i2{‘calldate’}’ AND recordingfile = '$i2{‘recordingfile’}'
LIMIT 3”;
print “\n$Mquery”;
my $numrows = $dbh->do($Mquery);
if ( not defined $numrows) {
print STDERR “ERROR: $DBI::errstr”;
} else {
print STDERR “\tINFO: $numrows rows updated”;
}
}
}
#print "$_ $info->{$_} " foreach keys %i2;
if ( &keystroke == 1 ) {
print “\nAbort. By ‘Q’”;
last;
}
}

$sth->finish();

$dbh->disconnect;

print “\nProcessed $NumRec records”;
exit 0;

sub mp3me{
my $DBH = shift;
my $File = shift;
my $recfile = shift;
my $calldate = shift;
my $lastapp = shift;
my $cnam = shift;
my $Desc = “$lastapp $cnam $calldate $recfile”;
$cnam =~ s/’//g;
$Desc =~ s/’//g;
my $Options = " --silent -m m --ta ‘$cnam PBX’ -b 16 --resample 44.1 “;
my $YEAROPT=”–ty " . substr($calldate,0,4);
my $Full = " $YEAROPT $Options --tt ‘$Desc’ --add-id3v2 $File";
my $NewFile = $File;
my ($Basename, $FullPath, $suffix) = fileparse($File , qr/.[^.]*/ );
$NewFile =~ s/.wav$/.mp3/;
die “Filename problem $NewFile $File” if $File eq $NewFile;
#$Full = " $File";
my ($atime, $mtime ) = (stat($File))[8,9];
print “\n\n $config{‘RMSILENCE’} $Full\n\n” if $DB;
print " rmsilence “;
system(”$config{‘RMSILENCE’} $File");
print “\n\n $config{‘LAME’} $Full\n\n” if $DB;
system("$config{‘LAME’} $Full “);
if ( -f $NewFile ){
print “\nSuccess. Lets move it to $config{‘archiveDir’}”;
system(“mkdir -p $config{‘archiveDir’}”);
#Change permissions on the newly minted .MP3 file
utime($atime,$mtime, $NewFile);
my $UID = getpwnam(‘root’) ;
my $GID = getgrnam(‘root’) ;
chown $UID, $GID, $NewFile;
chmod 0644 , $NewFile;
#Move the old wav file somewhere safe
move( $File , $config{‘archiveDir’} ) or die “Move did not succeed”;
die “No Recording files? Not possible” unless $recfile;
print " SQL”;
my $Mquery = “
UPDATE cdr
SET recordingfile = REPLACE(recordingfile, ‘.wav’, ‘.mp3’)
WHERE recordingfile = ‘$recfile’
”;
#WHERE calldate = ‘$calldate’ AND recordingfile = '$recfile’
my $numrows = $dbh->do($Mquery);
if ( not defined $numrows) {
print STDERR “ERROR: $DBI::errstr”;
}
} else {
die “\nFailure”;
}
print " … ";
#sleep 4 if -f ‘/tmp/lame-slow’ or $DB ;
}

sub keystroke {
my $i = ‘’;
vec($i, fileno(STDIN), 1) = 1;
my $j = select ($i, undef, undef, 0);
}

-That’s it in a nut shell. The Logs just show “1 file processed” because it isn’t an error. Any ideas would be appreciated.

-Christian

I’m looking for some help in trying to convert last years recordings from .wav to .mp3 (before i found the per-call-convert script).

I would like to convert the .wav to .mp3, update CDR so that the recordings are still accessible through CDR report and UCP. Once everything is completed, delete the .wav file. Also I’d like it to be such that i can specify which day’s (date) recordings i want to convert (one day at a time).

Thanks in advance for any help that I can get on this.

There are lots of scripts floating around to do this.

You might run into a problem on the UCP part, since I think it always looks for .wav files, but everything else is prior art - the script I’m running to do this had to be changed in 2010 to fix the fact that we assumed all of the years would start with 200…

I know I’ve posted my script for this years and years ago, and there have been lots of people posting similar scripts since. This isn’t an uncommon request - we have newbies log in here and ask for it all the time instead of searching through the archive of old posts.

I don’t think UCP will be a problem (atleast not in my case using FreePBX 13) since the converted files work fine for the script above.

I’ve been looking for different scripts but have always run into a road block. I don’t have enough scripting/programming knowledge. Would it be possible for you to share the script that you’re using?

I know I’m repeating myself but I need something that will convert files in batch, I don’t need something that will convert calls right after they’ve finished using mixmon/Post Call Recording Script.

Thanks!

If you record a lot of calls (like I do) using the post call recording option is a non-starter. PM me and I’ll send you the script I’m using. It’s not good form to post scripts here - too many things can go wrong with implementation and people just clog up the forum with support requests for long-dead topics.

Thanks for your help.

If you have lame, then

for i in /your/chosen/directory/*.wav; do nice -10 lame -b 320 -h “${i}” “${i%.wav}.mp3”; done

personally i would install the libsox-fmt-mp3 if you don’t have it and your sox version supports it so sox can do it and run it with

for i in /your/chosen/directory/*.wav; do nice -10 sox “${i}” “${i%.wav}.mp3”; done

prepend the call yo yor postrecordingscript with nice -n 10 and you shouldn’t have a problem.

2 Likes

Crickets

as always thanks for your input, but if I understand correctly this will not update the cdr database and hence my users will no longer be able to access their recordings (those that were converted with these commands) – is there a way to do both? convert old wav’s to mp3 and also update cdrdb?

you can add a command to the for loop between the ultlmate comma and done something like

mysql -ppassword -uuser asteriskcdrdb -e "update cdr set recordingfile='${i%.wav}.mp3' where recordingfile='$i'";

That might work, I havn’t tried it

This is probably more a question for dicko

Using information in one of your earlier posts, rather than running a cron job I am running my conversion scrip(s) to be called on by the utilizing the “Post Call Recording Script”, which I like a LOT more.

My question, since I am no where near proficient in Perl, is how do I remove my loop without stopping the rest of the script from running?

Here is a section I cut out that contains the loop, any ideas you could give me would be greatly appreciated!

if ( $Today eq $QDate )
{ print "sleeping ";sleep 10}
print " loop “;
my $NumRec = 0;
while( my $info = $sth->fetchrow_hashref){
$NumRec++;
my %i2 = %{$info}; #Easier to read
my @subdir = split( ‘-’, substr($i2{‘calldate’},0,10));
my $file = $config{‘monitor_base’} . ‘/’ . join (’/’ , @subdir) . “/$i2{‘recordingfile’}”;
$config{‘archiveDir’} = “/var/spool/asterisk/wavarchive/” . join (’/’ , @subdir) . ‘/’;
my ($archivo, $directorio, $suffix) = fileparse($file , ‘.wav’ );
printf “\n%5.0d %-60s " , $NumRec,”$archivo$suffix” ;#archive=$archivo directory=$directorio suffix=$suffix “;
next unless $suffix eq ‘.wav’;
#print " archive=$archivo directory=$directorio suffix=$suffix “;
if ( -f $file ){
mp3me($dbh,$file,$i2{‘recordingfile’},$i2{‘calldate’},$i2{‘cnam’},$i2{‘lastapp’});
print " Updated to MP3”;
} else {
print “Not Found! Duration $i2{‘duration’}”;
my $NewFile = $file;
$NewFile =~ s/.wav$/.mp3/;
if ( -f $NewFile ){
print “\n$file\n$NewFile\n OH an mp3 exists need to update the database record\n”;
my $Mquery = "
UPDATE cdr
SET recordingfile = REPLACE(recordingfile, ‘.wav’, ‘.mp3’)
WHERE calldate = ‘$i2{‘calldate’}’ AND recordingfile = ‘$i2{‘recordingfile’}’
LIMIT 3”;
print “\n$Mquery”;
my $numrows = $dbh->do($Mquery);
if ( not defined $numrows) {
print STDERR “ERROR: $DBI::errstr”;
} else {
print STDERR “\tINFO: $numrows rows updated”;
}
}
}

I’m not sure I ever posted a perl script, perhaps I did but bash is easily capable of doing anything you want, post both the script and the way you call it with all it’s parameter

Thanks for responding Dicko!
Originally I was trying to figure out why my script wouldn’t process more than one recording at a time when executed by a cron, but would process all available if I ran it manually. Using the GUI feature “Post Call Recording Script”, makes it run automatically after each call, which I think will be better, AND I won’t have to worry about solving the issue I have running it with a cron job.

I posted both the bash and perl scripts above on Jan 9th. It is really long so I don’t want to take up too much real estate by posting it again, but the script that starts the process is bash:

#!/bin/bash

TODAY=date +%F
echo “
–START-------------
”date >> /tmp/MP3convert.log
/usr/local/bin/wav2mp3/ls_rec_file.pl $TODAY | tee -a /tmp/MP3convert$TODAY.log | mail -r [email protected] -s “CES MP3 Convert” [email protected]

I did throw that in the Post Call box in the GUI and it ran great. Unfortunately I didn’t process them manually first, so there were about 250 wav’s to convert, and within a few seconds it had multiple instance trying to process all of the files at once… makes for an interesting HTOP to watch. Please note that I inherited the scripts and am just trying to make it work unattended for now, and later on I will try to come up with something much simpler to test on my test pbx.

Thanks for looking at this for me!