Archive for January, 2012

Creating a small (hopefully usable) utility: File Deleter

Wednesday, January 11th, 2012

When you are in media monitoring, you have TONS of files.  For example, look at this:

Multitude of files, StreamSink archive

Another bunch of files, created by PlayKontrol

Every recorder and logger and input produces a number of files on your system.  Of course, each application such as VideoPhill Recorder or StreamSink have the option for deleting a files after they expire (one month for example), but what if you have other way of gathering information (media, metadata, something) that won’t go away by itself?  I have several such data sources, so I opted to create a MultiPurposeHighlyVersatile FileDeleter Application.  I’ll probably find a better name later, for now lets call it ‘deleter’ for short.

The Beginning

The application to delete files must be such a triviality, I can surely open Visual Studio and start to code immediately.  Well, not really.  In my head, that app will do every kind of deleting, so let’s not get hasty, and let’s do it by the numbers.

First, a short paragraph of text that will describe the vision, the problem that we try to solve with the app, in few simple words.  That is the root of our development, and we’ll revisit it several times during the course of the development.

Vision:

‘Deleter’  should able to free the hard drive of staled files (files older than some period) and keep the level of hard drive space at some pre-determined minimum.

Here, it’s simple enough that I can remember it, and I’ll be able to descend down from it and create the next step.

The Next Step

For me, the next step (let’s say in this particular case) would be to try and see what ‘features’ does the app have.  The only way it works for me is to create a mock of the application UI and write down the things that aren’t visible from the UI itself.  Since this UI won’t do anything but gather some kind of parameters that will define behavior of the app, it will be a simple one, and it will be possible to fit it nicely on one screen.

For the sketch I’ll use Visual Studio, because I’m most comfortable with it.  If it wasn’t my everyday tool, I’ll probably use some application such as MockupScreens, which is completely trivialized app sketching gadget with powerful analyst features.

The process of defining the UI and writing down requirements took some time, I repeatedly added something to UI, then to the list below, until I had much clearer picture what I’m actually trying to do.

Features:

  • it should have ability to delete only certain files (defined by ‘mask’ such as *.mp3)
  • it should be able to delete file by their age
  • it should be flexible in determining the AGE of the file:
    • various dates in file properties: created, modified, accessed
    • by parsing the file name of the file
  • it should be able to delete from a multiple directories
  • it should be able to either scan directory as a flat or dig into subdirectories
  • it should be able to delete files by criteria other than age
    • files should be deleted if their total size exceeds some defined size
      • in that case, other files should be taken into account, again by mask
    • files should be deleted if minimum free drive space is less then some defined size
    • file size
  • when deleting by criteria other than file age, specify which files should be first to go
  • should be able to support multiple parameter sets at one time
  • should run periodically in predetermined intervals
  • should be able to load and save profiles
    • profiles should have names
  • should disappear to tray when minimized
  • should have lowest possible process priority
And here’s the screen:

Mock of the Deleter UI used to define and refine the requirements

As you look at the UI mock, you’ll see some mnemonic tricks that I use to display various options, for example:

  • I filled the textboxes to provide even more context to the developer (myself with another cap, in this case)
  • I added vertical scrollbars even if text in multi-line textboxes isn’t overflowing, to suggest that there might be more entries
  • for multiple choice options I deliberately didn’t use combobox (pull down menu) – I used radio button to again provide visual clues to various options without need for interaction with the mock

From Here…

I’ll let it rest for now, and tomorrow I’ll try to see if I can further nail down the requirements for the app.  From there, when I get a good feeling that this is something I’m comfortable with, I’ll create a object interface  that will contain all the options from the screen above.  While doing that, I’ll probably update requirements and the UI itself, maybe even revisit The Mighty Vision above.

BTW, it took me about 2 hours to do both the article and the work.  I excluded my wandering around time, of course :)

How to create a fair and adequate service proposal?

Thursday, January 5th, 2012

Since I’m about to create a media monitoring offer for my first end-user client of such kind, and as this caught me totally unprepared,  I’m on a journey of discovery for prices that would be fair to them but adequate to the company.

Media monitoring here is in a very limited context – they only need advertisement verification service, and that is the only service I can provide at the moment anyway.

For this estimation, I’ll try to use their side of view, and try to provide some added value.  At the end of this post, I’ll summarize the thoughts presented within.

The case

The prospect is a retail store company that has stores all through the country, and they are advertising on all media globally, and of course they use radio.

For every radio advertisement they have to pay some amount, defined by the stations’ price list, subject to various discounts, and so on.  For the payment they always get the invoice, and for most radio stations they also get ‘proof-of-playback’ document.

Proof-of-playback is usually generated from the automation software playout logs and processed with system such as SpotKontrol.  They are accurate most of the time, but sometimes, there are some discrepancies due to operator error or some other intricacy that’s going on.

Everything in the process is being done in a good will, but sometimes advertisements aren’t played and they are shown in the proof-of-playback document, and sometimes it’s the other way around.  Each playback is charged for some amount, and if the proof is incorrect, one party or the other is losing money.  It isn’t great situation for both of them.

So the idea would be to provide a service that could verify the document that is provided by the media by obtaining real and referent information on the playback of the advertisements.

Calculation

Some math and the abstract thinking would be required to read this section.  If you don’t mind reading it, just skip to the end where the results are shown.

In my estimation process I’ll always try to bound the numbers so they will show one extreme side of the possible cost range, and by doing so will come up with a cost that is always AT LEAST that amount.  For example, if there are 10 radio stations in question, and they have various cost of advertisement per second, I’ll use smallest number of them.

First estimation is that such a retail store will have AT LEAST 2 advertisements a day on AT LEAST 10 global radio channels.  We will also say that we will advertise only at workdays, so that gives us AT LEAST 20 days per month, giving us 2 * 10 * 20 = 400 advertisement playbacks.  That is the lowest bound, try to remember that, and we used only 10 global radio stations – most advertisers will go into local advertising as well.

Now, let’s try to estimate how much will each advertisement playback cost.  For that, we’ll use 5 prominnent radio stations and see their price lists.  We will also say that the advertisement in question will be AT LEAST 30″ in duration.  Radio stations:

The price for 30″ advertisement playback for those stations are: 360, 660, 110, 520, 400.  I would recommend Radio Istra to lift their prices of advertising up, and because of them I’ll go with next lowest price to be our estimate here: 360 kn.

From before, we had 400 advertisement playbacks per month, at a rate of 360 kn that amounts to 144.000 kn.  Since we promised we’ll use LOWEST bound, and some would be able to argue that there are various discounts that companies such as this can obtain, let’s say that the maximum amount of discount is 50%, and that will bring our cost down to half, and that is: 72.000 kn spent on advertising, each month.  In reality it is really a different number, but let’s go with this estimate here.

Now we know what are we insuring.  Let’s try to see what would be the cost of manually protecting that investment.

Let’s suppose that we have in place:

  • equipment to record and store 10 radio stations worth of broadcasting material (StreamSink for example)
  • means of reviewing (audibly) the archive of the broadcast material
  • a person that is trained to do all that.

I have the information that such person would cost about $12 in USA and about $4 to $6 in the cheap-labor countries.  Let’s say that our guy will cost $8 = about 50 kn per hour.

From my experience, and by using VideoPhill Player to access the archive, confirming 20 advertisements playback would last about one hour if we do have a proof-of-playback document, and at least 2 hours if we don’t, since whole block of advertisements would have to be under scrutiny.  Also, here we assume that our operator is HIGHLY familiar with scheduling practices of each radio station, and won’t stray too much while searching for the advertisement blocks.

So with all the equipment, and trained staff, it seems that cost of verification for that kind of volume is from 1000 kn to 2000 kn per month.  If we use average number here, and see what’s the ratio of the analyst cost per investment that he protects, we come up with 1:48 or 2%.


Result: we can say that cost of verification for advertisements is about 2% of the cost of the advertising.

It would be even higher (in percentage) if we were considering other media that has lower cost of advertising, since operator would have to scan (at same cost) the material that is paid less.

Conclusion

Since I’m not here to promote the service that isn’t needed by someone, I’ll only try to provide fair price for it for someone that recognizes the need for it.  To do that, I’ll go with 50% discount on my already low estimation, and will try to see if the company can sustain that service at that fee.

That being said, the conclusion is that

to provide a list of played commercials we’ll charge 1% of estimated monthly advertisement cost for that channel(s).

We’ll start from there, and see where it takes us. :)

Testing 3rd party stream capture application

Monday, January 2nd, 2012

This is a response to a question from one of my prospects, and it can be summarized as:

Why should I buy StreamSink at $10.000
when there is Replay A/V that can do same
thing for $100 (if I buy 2 licences for 
2 computers)?

I can make several objections to the idea of having a consumer product in use for business purpose, but instead of that, I’ll try to focus on functionality (at least for this posting).

Purchasing and installing

I quickly purchased Replay A/V for $50, and went on to installing it.  Upon installation, it offered to install WinPopcap (to provide stream discovery) and some other utility for conversion of the saved material.  I declined.

Entering stations

Once installed, I will try to copy my stream list into it and have it record it continuously.

After some investigation, I found out that there is no way to insert the list of the stations at once, so I’m going to enter them one by one.

OK, I entered Antena Zagreb with its stream URL, and went on to fiding the start button for it.  I found it under context menu for the item that was on the list (right click, start-recording, …).

I remembered that I went through the options for a channel and found that you have to explicitly have to enter the option for splitting the file into segments, so I went on and did that.

I’ll leave it run now and will move on to enter the rest of the stations.

I was about 1/4 way down the list, then I got to WMA stream, and was really curious whether it will be accepted, since there is nowhere a option to pick a stream type.  It was, and for now, it seems that it’s captured normally.

When I am entering the data into the software, and it does its file splitting at every 5 minute intervals, whole GUI freezes and becomes unresponsive for 2-3 seconds.  What I am interested in is whether there will be a gap in the recording of the station that is cut.  BTW, the computer I am doing the analysis at isn’t so weak…

Also, it seems that I entered a stream that doesn’t exists.  Application is persistent in trying to connect to it, but while doing so, it freezes again for few seconds.  However, it’s nothing to be alarmed about.

I also found out that in order for the app to be persistent about recurrent connecting, it has to be additionally configured, as it is not the default mode of the operation.

OK, so I finally entered all the stations.  It gets rather annoying after few minutes, because on the 5 minute chunk interval, app gets its freezing moments rather frequently, and despite the fact it doesn’t pose a problem AFTER everything is entered, it really is annoying.  Here is the filled up application:

Testing the recorded stuff

To do that, I will first share the folder with recordings so I would be able to see it from another (this) machine.

As expected, every channel is saved in its appropriate folder:

Now, let’s examine the contents of some folders that are recorded here…

First folder I have is Antena Zagreb, and here it is:

I won’t comment file naming now, but will tell you what happened when I double-clicked .m3u file that should have the list of mp3 files that are recorder. Winamp loaded it and CRASHED my machine completely. I don’t say it will crash yours, but my Winamp, when faced with certain media files that it can’t recognize, goes berserk. The problem here lies in the fact that Antena Zagreb has AACPLUS stream, and it was interpreted erroneously, creating mp3 files that crashed the Winamp. Here is one file for you to try, use it on your own risk.

Antena Zagreb Jan 02_05

Media Player crashed as well, but I could END it, with Winamp I had to restart the whole machine.

Last test I want to do in this post is to see if the subsequent files are saved so there is no gap between.  For that, I have to find a mp3 file that won’t actually break my player.

Found it, and had no luck.  Even with pure mp3 files, Winamp gives up and puts its legs in the air.  Tested the same with Media Player, and it seems that recordings overlap by few seconds, so that checks out.

Before conclusion, let’s just take a look at resource usage of the application:

Conclusion

You might be able to use Replay A/V for your media monitoring purposes, and save great deal of money.  However, please note that:

  • I didn’t find any option for error reporting (which will enable you to see that the stream is off-line for extended time)
  • if all the channels would cut the file at the same time, it would create unresponsive app for at least 2*number_of_channels seconds
  • CPU usage profile is minimal, however I just found out that memory usage rises LINEARLY over time, and that would lead to immanent application death after some time (you do the math)
  • I didn’t use scheduler to create persistent connections, if I would, and am having bad connection with lots of breaks, app would be nearly impossible to use due to freezing upon connection
  • there is no (or I wasn’t able to find it) option for renaming the files so they would use some time-stamped names
  • it doesn’t provide support for VideoPhill Player, which is a archive exploration tool created just for Media Monitors

Additional info…

After several hours (around 6) this is the memory usage that is taken using Procexp.

For those that can’t read memory usage graph, this means that the application has a memory leak, and by this rate, it would exhaust its memory in less then 24 hours, since it is x86 process.  Quick remedy for that would be to raise the interval for the file cutting, because I suspect that memory leak occurs at that time.