Converting recorded wav calls to mp3

I have been convertine wav’s to MP3’s for years and saving to an external storage server using a fuse mounted directory, under the “monitor” directory.
Recently I switched to a new-build FreePBX14 (official distro), which I love by the way, but now my scripts are buggy and I am hoping that someone in here will see something that I am missing, and help be get it working correctly…

The problem isn’t huge, but a real PITA because I have to remember to maually execute the job late in the day, or it won’t finish before midnight, and then I have un-converted files.

The Problem is simply:
When step 1 (MP3convet.sh) is executed by crontab, it processes ONE file instead of running all available.
If I manually execute /usr/local/bin/MP3convert, it will run everything available at the time I execute it.

The whole process takes about 12 seconds including logging into the db, loggingq and reporting

Basically I have a .sh that starts the main script, then reports to a couple email addresses:

Step 1 (crontab): */05 * * * * /usr/local/bin/wav2mp3/MP3convert.sh (FYI - I tried 10 minutes, 5 minutes, and 1 minute)

Step2: MP3convert.sh

#!/bin/bash

TODAY=date +%F
echo “
–START-------------
date >> /tmp/MP3convert.log
/usr/local/bin/wav2mp3/ls_rec_file.pl $TODAY | tee -a /tmp/MP3convert$TODAY.log | mail -r [email protected] -s “CES MP3 Convert” [email protected]

Step 3: The mack-daddy which handles time stamp, remove silence, convert to mp3, and update CDR to reflect wav to mp3 extension. (note that I added some dots [ ie .#] to stop the post from having bold letters)

#!/usr/bin/perl -w

.# Script to post process recordings in Asterisk / FreePBX

.# You can set in FreePBX the “Run After Record” parameter
.# to be:

.# /var/lib/asterisk/bin/record_runafter.pl ^{UNIQUEID} ^{MIXMONITOR_FILENAME}

$|++;
use strict;
use DBI;
my %config;
my $dbh;
use File::Basename;
use File::Copy;

my $DB = $ENV{‘DEBUG’};

#Get the date from the environment OR ARG[0]
my $QDate = $ENV{‘QDATE’} || $ARGV[0];
die “$0: QDATE not set nor ARG1 set.” unless $QDate;

.# Destination Directory for recordings
$config{‘destDir’} = “/var/spool/asterisk/monitor/”;

.# What app do we use to encode?
$config{‘LAME’} = which lame;
chomp($config{‘LAME’});
die "$0: Lame not found " unless $config{‘LAME’};

#What app do we use to remove silence
$config{‘RMSILENCE’} = ‘/usr/local/bin/wav2mp3/rmsilenceWAV.sh’;
die “$0: $config{‘RMSILENCE’} not found.” unless -x $config{‘RMSILENCE’};

.# Read FreePBX configuration
open( CONFIG, “</etc/amportal.conf” ) or die(“Could not open /etc/amportal.conf. Aborting…”);
while () {
chomp;
$_ =~ s/^\s+//g;
$_ =~ s/([^#]*)#/$1/g;
$_ =~ s/\s+$//g;
if ($_ ne “”) {
my($key,$val) = split(/=/);
$config{$key}=$val;
}
}

$config{‘cdrHost’} = $config{‘CDRDBHOST’} ?$config{‘CDRDBHOST’}:$config{‘AMPDBHOST’};
$config{‘cdrDBName’} = ‘asteriskcdrdb’;
$config{‘cdrTableName’} = $config{‘CDRDBTABLENAME’} ?$config{‘CDRDBTABLENAME’}:‘cdr’;
$config{‘cdrUser’} = $config{‘CDRDBUSER’} ?$config{‘CDRDBUSER’}:$config{‘AMPDBUSER’};
$config{‘cdrPass’} = $config{‘CDRDBPASS’} ?$config{‘CDRDBPASS’}:$config{‘AMPDBPASS’};
$config{‘monitor_base’} = $config{‘MIXMON_DIR’} ? $config{‘MIXMON_DIR’} : $config{‘ASTSPOOLDIR’} . ‘/monitor’;

#my $uniqueid = $ARGV[0];
#my $directorio_archivo = $ARGV[1];
#my $type = $ARGV[2];

print “\n$=$config{$}” foreach qw/ cdrHost cdrDBName cdrTableName cdrUser cdrPass AMPDBHOST /;

sub connect_db() {
my $return = 0;
my %attr = (
PrintError => 0,
RaiseError => 0,
);
my $dsn = “DBI:mysql:database=$config{‘cdrDBName’};host=$config{‘AMPDBHOST’}”;
$dbh->disconnect if $dbh;
$dbh = DBI->connect( $dsn, $config{‘cdrUser’}, $config{‘cdrPass’}, %attr ) or $return = 1;
return $return;
}

my $time = localtime(time);
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localtime(time);
$mon++;
$year += 1900;

#Change upper time limit to 2 minutes back
my $Today = sprintf ("%4d-%02d-%02d",$year,$mon,$mday);
if ( $min < 2 ){
$hour–;
$min = ‘58’;
} else {
$min–;
}

my $Date2 = $Today . " " . sprintf("%02d:%02d:%02d",$hour,$min,‘0’) ;

# Connect to Database

&connect_db();

print “\nTODAY=$Today QDate=$QDate through $Date2”;

print “\nStarting …”;
my $query = “SELECT calldate,recordingfile,duration,cnam,lastapp
FROM $config{‘cdrTableName’}
WHERE DATE(calldate) = ‘$QDate’ AND calldate < ‘$Date2’ AND recordingfile > ’ ’ AND SUBSTRING(recordingfile, -4) = '.wav’
ORDER BY calldate
limit 4000”;

my $sth = $dbh->prepare($query) or die “SQL-do Error: $DBI::errstr\n”;
$sth->execute or die “SQL Error: $DBI::errstr\n”;

#Let recordings settle before we start if we are processing Today
sleep 10 if $Today eq $QDate;

my $NumRec = 0;
while( my $info = $sth->fetchrow_hashref){
$NumRec++;
my %i2 = %{$info}; #Easier to read
my @subdir = split( ‘-’, substr($i2{‘calldate’},0,10));
my $file = $config{‘monitor_base’} . ‘/’ . join (’/’ , @subdir) . “/$i2{‘recordingfile’}”;
$config{‘archiveDir’} = “/var/spool/asterisk/wavarchive/” . join (’/’ , @subdir) . ‘/’;
my ($archivo, $directorio, $suffix) = fileparse($file , ‘.wav’ );
printf “\n%5.0d %-60s " , $NumRec,”$archivo$suffix" ;#archive=$archivo directory=$directorio suffix=$suffix ";
next unless $suffix eq ‘.wav’;
#print " archive=$archivo directory=$directorio suffix=$suffix “;
if ( -f $file ){
mp3me($dbh,$file,$i2{‘recordingfile’},$i2{‘calldate’},$i2{‘cnam’},$i2{‘lastapp’});
print " Updated to MP3”;
} else {
print “Not Found! Duration $i2{‘duration’}”;
my $NewFile = $file;
$NewFile =~ s/.wav$/.mp3/;
if ( -f $NewFile ){
print “\n$file\n$NewFile\n OH an mp3 exists need to update the database record\n”;
my $Mquery = “
UPDATE cdr
SET recordingfile = REPLACE(recordingfile, ‘.wav’, ‘.mp3’)
WHERE calldate = ‘$i2{‘calldate’}’ AND recordingfile = '$i2{‘recordingfile’}'
LIMIT 3”;
print “\n$Mquery”;
my $numrows = $dbh->do($Mquery);
if ( not defined $numrows) {
print STDERR “ERROR: $DBI::errstr”;
} else {
print STDERR “\tINFO: $numrows rows updated”;
}
}
}
#print "$_ $info->{$_} " foreach keys %i2;
if ( &keystroke == 1 ) {
print “\nAbort. By ‘Q’”;
last;
}
}

$sth->finish();

$dbh->disconnect;

print “\nProcessed $NumRec records”;
exit 0;

sub mp3me{
my $DBH = shift;
my $File = shift;
my $recfile = shift;
my $calldate = shift;
my $lastapp = shift;
my $cnam = shift;
my $Desc = “$lastapp $cnam $calldate $recfile”;
$cnam =~ s/’//g;
$Desc =~ s/’//g;
my $Options = " --silent -m m --ta ‘$cnam PBX’ -b 16 --resample 44.1 “;
my $YEAROPT=”–ty " . substr($calldate,0,4);
my $Full = " $YEAROPT $Options --tt ‘$Desc’ --add-id3v2 $File";
my $NewFile = $File;
my ($Basename, $FullPath, $suffix) = fileparse($File , qr/.[^.]*/ );
$NewFile =~ s/.wav$/.mp3/;
die “Filename problem $NewFile $File” if $File eq $NewFile;
#$Full = " $File";
my ($atime, $mtime ) = (stat($File))[8,9];
print “\n\n $config{‘RMSILENCE’} $Full\n\n” if $DB;
print " rmsilence “;
system(”$config{‘RMSILENCE’} $File");
print “\n\n $config{‘LAME’} $Full\n\n” if $DB;
system("$config{‘LAME’} $Full “);
if ( -f $NewFile ){
print “\nSuccess. Lets move it to $config{‘archiveDir’}”;
system(“mkdir -p $config{‘archiveDir’}”);
#Change permissions on the newly minted .MP3 file
utime($atime,$mtime, $NewFile);
my $UID = getpwnam(‘root’) ;
my $GID = getgrnam(‘root’) ;
chown $UID, $GID, $NewFile;
chmod 0644 , $NewFile;
#Move the old wav file somewhere safe
move( $File , $config{‘archiveDir’} ) or die “Move did not succeed”;
die “No Recording files? Not possible” unless $recfile;
print " SQL”;
my $Mquery = “
UPDATE cdr
SET recordingfile = REPLACE(recordingfile, ‘.wav’, ‘.mp3’)
WHERE recordingfile = ‘$recfile’
”;
#WHERE calldate = ‘$calldate’ AND recordingfile = '$recfile’
my $numrows = $dbh->do($Mquery);
if ( not defined $numrows) {
print STDERR “ERROR: $DBI::errstr”;
}
} else {
die “\nFailure”;
}
print " … ";
#sleep 4 if -f ‘/tmp/lame-slow’ or $DB ;
}

sub keystroke {
my $i = ‘’;
vec($i, fileno(STDIN), 1) = 1;
my $j = select ($i, undef, undef, 0);
}

-That’s it in a nut shell. The Logs just show “1 file processed” because it isn’t an error. Any ideas would be appreciated.

-Christian

I’m looking for some help in trying to convert last years recordings from .wav to .mp3 (before i found the per-call-convert script).

I would like to convert the .wav to .mp3, update CDR so that the recordings are still accessible through CDR report and UCP. Once everything is completed, delete the .wav file. Also I’d like it to be such that i can specify which day’s (date) recordings i want to convert (one day at a time).

Thanks in advance for any help that I can get on this.

There are lots of scripts floating around to do this.

You might run into a problem on the UCP part, since I think it always looks for .wav files, but everything else is prior art - the script I’m running to do this had to be changed in 2010 to fix the fact that we assumed all of the years would start with 200…

I know I’ve posted my script for this years and years ago, and there have been lots of people posting similar scripts since. This isn’t an uncommon request - we have newbies log in here and ask for it all the time instead of searching through the archive of old posts.

I don’t think UCP will be a problem (atleast not in my case using FreePBX 13) since the converted files work fine for the script above.

I’ve been looking for different scripts but have always run into a road block. I don’t have enough scripting/programming knowledge. Would it be possible for you to share the script that you’re using?

I know I’m repeating myself but I need something that will convert files in batch, I don’t need something that will convert calls right after they’ve finished using mixmon/Post Call Recording Script.

Thanks!

If you record a lot of calls (like I do) using the post call recording option is a non-starter. PM me and I’ll send you the script I’m using. It’s not good form to post scripts here - too many things can go wrong with implementation and people just clog up the forum with support requests for long-dead topics.

Thanks for your help.

If you have lame, then

for i in /your/chosen/directory/*.wav; do nice -10 lame -b 320 -h “${i}” “${i%.wav}.mp3”; done

personally i would install the libsox-fmt-mp3 if you don’t have it and your sox version supports it so sox can do it and run it with

for i in /your/chosen/directory/*.wav; do nice -10 sox “${i}” “${i%.wav}.mp3”; done

prepend the call yo yor postrecordingscript with nice -n 10 and you shouldn’t have a problem.

2 Likes

Crickets

as always thanks for your input, but if I understand correctly this will not update the cdr database and hence my users will no longer be able to access their recordings (those that were converted with these commands) – is there a way to do both? convert old wav’s to mp3 and also update cdrdb?

you can add a command to the for loop between the ultlmate comma and done something like

mysql -ppassword -uuser asteriskcdrdb -e "update cdr set recordingfile='${i%.wav}.mp3' where recordingfile='$i'";

That might work, I havn’t tried it

This is probably more a question for dicko

Using information in one of your earlier posts, rather than running a cron job I am running my conversion scrip(s) to be called on by the utilizing the “Post Call Recording Script”, which I like a LOT more.

My question, since I am no where near proficient in Perl, is how do I remove my loop without stopping the rest of the script from running?

Here is a section I cut out that contains the loop, any ideas you could give me would be greatly appreciated!

if ( $Today eq $QDate )
{ print "sleeping ";sleep 10}
print " loop “;
my $NumRec = 0;
while( my $info = $sth->fetchrow_hashref){
$NumRec++;
my %i2 = %{$info}; #Easier to read
my @subdir = split( ‘-’, substr($i2{‘calldate’},0,10));
my $file = $config{‘monitor_base’} . ‘/’ . join (’/’ , @subdir) . “/$i2{‘recordingfile’}”;
$config{‘archiveDir’} = “/var/spool/asterisk/wavarchive/” . join (’/’ , @subdir) . ‘/’;
my ($archivo, $directorio, $suffix) = fileparse($file , ‘.wav’ );
printf “\n%5.0d %-60s " , $NumRec,”$archivo$suffix” ;#archive=$archivo directory=$directorio suffix=$suffix “;
next unless $suffix eq ‘.wav’;
#print " archive=$archivo directory=$directorio suffix=$suffix “;
if ( -f $file ){
mp3me($dbh,$file,$i2{‘recordingfile’},$i2{‘calldate’},$i2{‘cnam’},$i2{‘lastapp’});
print " Updated to MP3”;
} else {
print “Not Found! Duration $i2{‘duration’}”;
my $NewFile = $file;
$NewFile =~ s/.wav$/.mp3/;
if ( -f $NewFile ){
print “\n$file\n$NewFile\n OH an mp3 exists need to update the database record\n”;
my $Mquery = "
UPDATE cdr
SET recordingfile = REPLACE(recordingfile, ‘.wav’, ‘.mp3’)
WHERE calldate = ‘$i2{‘calldate’}’ AND recordingfile = ‘$i2{‘recordingfile’}’
LIMIT 3”;
print “\n$Mquery”;
my $numrows = $dbh->do($Mquery);
if ( not defined $numrows) {
print STDERR “ERROR: $DBI::errstr”;
} else {
print STDERR “\tINFO: $numrows rows updated”;
}
}
}

I’m not sure I ever posted a perl script, perhaps I did but bash is easily capable of doing anything you want, post both the script and the way you call it with all it’s parameter

Thanks for responding Dicko!
Originally I was trying to figure out why my script wouldn’t process more than one recording at a time when executed by a cron, but would process all available if I ran it manually. Using the GUI feature “Post Call Recording Script”, makes it run automatically after each call, which I think will be better, AND I won’t have to worry about solving the issue I have running it with a cron job.

I posted both the bash and perl scripts above on Jan 9th. It is really long so I don’t want to take up too much real estate by posting it again, but the script that starts the process is bash:

#!/bin/bash

TODAY=date +%F
echo “
–START-------------
”date >> /tmp/MP3convert.log
/usr/local/bin/wav2mp3/ls_rec_file.pl $TODAY | tee -a /tmp/MP3convert$TODAY.log | mail -r [email protected] -s “CES MP3 Convert” [email protected]

I did throw that in the Post Call box in the GUI and it ran great. Unfortunately I didn’t process them manually first, so there were about 250 wav’s to convert, and within a few seconds it had multiple instance trying to process all of the files at once… makes for an interesting HTOP to watch. Please note that I inherited the scripts and am just trying to make it work unattended for now, and later on I will try to come up with something much simpler to test on my test pbx.

Thanks for looking at this for me!

Hey dicko,
Forgive me as I’m not well versed in handling audio files on linux. Are the methods used in the scripts in this thread using the lame package to convert to MP3 and you are saying you use the sox package (complied with liblame) to perform the conversion? And this conversion is quicker? And you would change the scripts from running;

nice lame -b 32 -m m "$dtpath$1.wav" "$dtpath$1.mp3"

to a command using sox instead?

Thanks.

Using sox with soxlib-fmt-mp3 installed instead of lame, just replace the call to lame with one to sox.

sox “$dtpath$1.wav” “$dtpath$1.mp3”

Thanks. I found there is a settings in FreePBX to change the recordings from wav to WAV (WAV49) and am considering just doing that instead. Seems simpler and probably negligible difference in storage savings to mp3 (for me at least).

I’m seeing two issues currently.

  • The “call recording report” in FreePBX is showing the “duration” incorrectly now that it’s set to WAV49 (a 3min call showed 6sec). The playback works fine though.
  • CPU spikes to 99% for a 1-2sec. I’d assume it was the conversion causing it but what I see is a mysql PID. I haven’t seen this behavior before and I’m thinking perhaps it is related to the recording in some way.

Well, choose a good value for nice and it should play well with the other kids, mp3 wav WAV etc. have in the past been problematic as android iphones windoze ios etc. could never all agree on your choice, hence voicemail will record many at your behest, I personally have moved to ogg vorbis ( sox supported) as anything that understands html5 can read it., some old sip phones still only read .wav though. Call .ogg the .wav of the future?

I’m talking about the “call recording format” option in Advanced Settings. I changed that to WAV. No ogg/mp3 option there. So no need to set a script to run.

Considering WAV49 is pretty space efficient and this method is really straight forward, what am I missing that makes this less ideal of a solution then a script calling SOX to convert to mp3, ogg, or whatever? This method I don’t have to bother updating mysql in a script file either.

If none of your clients complain that they can’t play wav49, then you are done.

Unlikely to be able to play that on many devices from my experience so check yours - and it can’t be converted to mp3 as well was the PCM encoded wav from my experience - i.e. if you are happy with that format great, but don’t count on it being suitable for format conversion later. The mp3 that result will be of poor quiality.