Suggestions for large volume call-recording

General Help
1

We are currently monitoring all of the calls that come in to our main call queue. This works quite well, but it is resulting in a huge number of files and related storage, organization, and management problems.

We have been recording calls on our current system since May of 2007, resulting in 233000+ call monitor files. This causes the following problems:

1.) It takes forever for amportal to start up asterisk, as it hangs for a few minutes on the “Check File Permissions” step. (That’s a lot of permissions to check!)
2.) It takes forever to use the Trixbox CDR report to find calls, as it generates links to each recorded file.
3.) I am eventually going to run into some sort of filesystem issue, though this is less of a problem as it would have been on ext2 or some other filesystem.
4.) Although it might be unrelated, ARI does not work on my system, as the call monitor links do not appear, even though the UNIQUEID is stored in the Database.

Does anyone have any suggestions as to how I might arrange these files to mitigate these problems? I could easily move files manually into directories based on year, month, etc., but that would break the functionality built-in to ARI and the Trixbox CDR report that allow me to look up a call by CDR and click on a link to listen to a call recording.

Any thoughts?

Tom

2

The first question I think you need to ask yourself is do you REALLY need to keep the call of employee xx boss talking to his wife from almost two years ago?

Next so that you know the trixbox cdr call reporting interface where you can clink on it and get the audio file is actually a huge security issue as it does directory traversal and anybody who goes digging around can find ways to suddenly gain access to parts of your phone system you probably think are secured. That has a lot to do with why it’s only available on trixbox and not a built in feature of FreePBX.

Don’t you think that it is time to take all files after xxx days old and move them into a archive system of some form so that it does not effect the performance of the system.

3

Well, fskrotzki, as you may remember from earlier threads about ARI and call recordings, etc, I am actively seeking to dump our older Trixbox install. Security problems, as you note, are a major reason for this desire. Also from earlier threads, ARI is not designed in such a way to reasonably sort through large numbers of recordings, thus my use of the Trixbox CDR report.

As to the question of whether I need to keep old recordings, the only recordings we keep are incoming queue calls, and though there are some that I could get rid of, there is no good way to segregate the necessary and unnecessary. More to the point, I do need to dig up calls from years past on a surprisingly regular basis.

As to your question about archiving, I indeed do think it is time to move the files into some sort of archive based on date; that’s exactly why I started this thread. What I am looking for are suggestions as to how I can archive the calls and still be able to retrieve them when viewing CDR records. Unfortunately, it seems that the existing tools (ARI, Trixbox/Elastix CDR report, modifications to FreePBX’s CDR pages, etc) will all break if the files are not found in the /var/spool/asterisk/monitor directory (or whatever directory you specify in the tool’s setup).

Maybe I’m the only FreePBX customer who has these needs, but I doubt it. To me, it seems that FreePBX should have a way to do the following:

1.) Easily find call monitor recordings, even if you have thousands of them. ARI is partially there, but the search function is vague (ie: no specific date, destination, etc fields, and it’s not always obvious what format you need to use when searching 1/2/2004? 1-2-2004? jan 2 2004? which?) Sure I can figure it out (and would have if it worked on our system), but why the unintuitive interface?
2.) Call recordings should have some sort of built-in archiving solution to handle the inevitable large number of recordings for any user who has any volume of call recordings.

My $0.02. I think this would be of benefit to the community as a whole, because I am sure that there are any number of customers who have had to develop their own solutions to these problems. hell, I could even fund development of a module that would handle these needs.

Tom

4

Now that you mention it I do remember your posts. I didn’t put two and two together when I read the post the first time.

We have call recording as a option for our company to use when you are on a call but I think in the last two years it’s been used 16 times total.

So needing it and how it works is really pretty subjective based on who you are, what type of business you are in and if it has requirements for it or not.

The problem with implementing an archiving solution is that it will place extra set of demands/requirements on the system and the development team tries real hard to not do that. Then there is the issue of standards and compliance, where is it needed and who’s rules are you going follow. It get’s real ugly at that point. Financial organizations for example have very specific requirements both at a government level and here in the US (where I’m at) sometimes at the state level. The moment you throw in international requirements that’s like several peoples full time job to just track all the requirements down. So I think that has a lot to do with why it’s not touched.

I bet it would be possible to ask for development of a common API so that you can retrieve a call record (and recording), and another call to delete one, that might happen. What you do with it past that point really going to be up to you because like I said the requirements are way to diverse to attempt to match based on industry, county and potential local codes.

It might be possible to create a generic system around the general idea but for me personally I’d want that archive system to be a external to our current phone system, I did way over spec the box so that we’d not have issues for years but it’s not configured for maintaining that kind of volume of data and I’'l bet most are not.

5

I definitely agree that the need for recording is subjective. I would suggest that there is little needed with regard to determining when a call is recorded. The current tools (queue setup, extension setup, etc) work well.

However, what I think is universally needed for any user that records more than a few calls per year is an interface that allows the user to easily find and listen to those recordings. The ideal solution in my mind is to add a link to the recording file in the output of the CDR search. Current implementations (see http://www.freepbx.org/trac/ticket/318 ), tend to slow to a crawl when dealing with a large number of files because they search for a recording file for each record returned.

Additionally, something should be available to archive call recordings, such as a folder for each year, and within that, a folder for each month. This ought to be sufficient for most implementations, because state, federal, and international requirements for call recording will specify how long you must keep them, not what the directory structure should look like. Importantly, moving these files to archive folders should not break the tool I describe above.

On a side note, high volume call recording doesn’t require a heavy-duty machine, as we run our system on a Pentium D, 2GB RAM, and our 2 years of recordings still only take up 62GB of space in WAV49 format.

6

You might want to give this application a try - haven’t tried it myself, so let us know if it works

http://www.virtutel.com/archiver.html

Mag