VIEWS: 14 PAGES: 10 POSTED ON: 4/24/2011
Record Noise Filtering Marc Kessler 4/25/05 Overview: The purpose of this project is to determine applicable methods for noise removal and crackling from old LP audio. The two methods tested in this project are both non- linear. The double median filter is a good method for the removal of outlying data spikes from a trend. A large factor determining the effectiveness of the filter is the number of samples used to compute the median. If too small of a number is chosen, the median will capture the impulsive spikes, and if too large of a number is chosen, the sound will distort. The second method used to improve the audio is a high-frequency spectral noise gate. This non-linear filtering method removes any frequency components with magnitudes less than a threshold value. The threshold value determines if the entire signal is removed, passed, or has components removed. Determining a good method for the dynamic alteration of this value is very important to the functionality of this noise removal scheme. Method: Median Filter: The median filter removes crackling from the input signal. Each media filter is essentially an average that ignores spikes in the signal. As a result it behaves as a low-pass filter. The double median filter (figure 1a) passes the signal through the first median which removes the high frequency and noise components. It then subtracts that from the original signal leaving only the high frequency and noise. This is then filtered again which leaves the high frequency signal with the noise removed. The filter finally combines both the high and low frequency components to reconstruct the original signal with impulses and trend deviations removed. Because the median filter takes the median of the nth to n-x samples a buffer needs to be created to store these samples. Values of x from 8 to 14 seemed to provide good results. Choosing too small of a value kept most of the crackles and noise, and choosing too large of a value caused a lot of distortion to the signal. x[n] + y[n] Median + - Median + Figure 1a – Median Filter Block Diagram Noise-Gate Filter: The noise-gate filter tries to remove noise from a signal by removing low amplitude frequency components. It does this by windowing the input signal, taking the FFT, setting any frequency values below a threshold value to 0, then computing the Inverse-FFT and recombining the windowed sections. The code was tested both with and without windowing the input signal. It was determined that windowing provided superior results by creating seamless transitions between the sampled segments. The noise-gate filter relies heavily upon a threshold value to determine which frequencies to keep and which to drop. A basic algorithm was implemented to determine a reasonable threshold value for each sampling segment. The algorithm looks at the power spectrum of the 256 sample FFT and assumes the signal will contain frequencies between 700Hz and 2500KHz (values can be specified in the header of the program). It then determines the median of the “guaranteed” power spectrum, the range of 700Hz to 2500KHz, and the minimum value of the entire spectrum. The threshold value for comparison is 2% of the difference of the median of the guaranteed spectrum and the minimum value added to the minimum value. To further prevent distortion to the signal, only frequencies above 7KHz will be dropped by this method. Although this method can’t remove the lower frequency components of white noise, it protects the signal from loosing lower frequency definition. This process is illustrated in figure 1b. The output of the noise-gate is the same as the median filtered input, but most of the remaining high frequency noise is removed. Figure 1b – Spectral Noise-Gate Filter Method Implementation: A basic code outline was written to quickly test the implementation of the median filter and spectral noise-gate. Once this was completed, variables were placed so that the behavior of each filter could be altered and then many audio samples were gathered with different filter configurations. It was determined that the median filter should be run before the noise-gate filter because the median filter left some impulsive, high-frequency artifacts, which were smoothed by the noise-gate. Figure 2 shows the progression of the signal through the filtering program. The median filter removes a lot of the clicks and some noise. The window makes a smooth transition between segments used for the FFT. The FFT and IFFT are required for the spectral noise gate, which will remove more of the high-frequency noise. x[n] Hanning Median Filter FFT Window IFFT Spectral Filter y[n] Figure 2 Figure 3 and 4 shows the original signal on the top channel and the cleaned signal on the lower channel. One can see from the entire signal that most of the impulsive clicks have been removed. With a closer look provided by figure 4 it can be seen how noise in the wave has been reduced, producing a smooth, cleaner wave. Figure 3 Figure 4 Figures 5 and 6 show a comparison of the filtered signal to the “ideal” signal. The output of the noise removal is very similar to the ideal output, but some clicks and hissing remain. The cleaned signal is a great improvement from the initial input, nearly matching the ideal signal, demonstrating the effectiveness of noise removal scheme. Figure 5 Figure 6 Figure 7 shows the progression of the signal through each filter and transform. These waveforms are from an implementation prior to windowing. Good results are provided, but windowing provides for cleaner sample transitions. It can clearly bee seen how the median filter smoothes the signal and removes most spikes. It can also be seen how the frequency decimation removes the remaining noise and unwanted high frequency components. At this point in the code development the input and clipped spectrums were on different scales. Input Median Output 1000 400 200 500 0 0 -200 -500 -400 -1000 -600 0 50 100 150 200 250 300 0 50 100 150 200 250 300 4 4 x 10 Input Spectrum x 10 Clipped Spectrum 2.5 2.5 2 2 1.5 1.5 1 1 0.5 0.5 0 0 0 50 100 150 200 250 300 0 50 100 150 200 250 300 Spectrum Output 400 200 0 -200 -400 -600 0 50 100 150 200 250 300 Figure 7 Figure 8 shows the same filter progressions as figure 7, but the signal is windowed to get a smooth transition between sample sets. Because of this windowing, the centered data has large amplitudes and as you approach the left or right side the signal tends toward zero. 4 x 10 Input Median Output 1.5 10000 1 5000 0.5 0 0 -0.5 -1 -5000 0 100 200 300 0 100 200 300 5 5 x 10 Input Spectrum x 10 Clipped Spectrum 3 3 2 2 1 1 0 0 0 100 200 300 0 100 200 300 Spectrum Output 10000 5000 0 -5000 0 100 200 300 Figure 8 Improvements: There are several improvements that can be made to improve the noise filter. Setting frequency values to 0 in the FFT causes ripples in the time domain when the IFFT is used. To reduce this, a smoother filter could be used. There are also several values that were chosen based upon performance which don’t adapt automatically to the signal. The threshold value has a predetermined scaling factor of 2%. A smarter scheme may be able to adapt this value to better fit the incoming signal. Another fixed value is the samples used to compute the median. An adaptive method based on the derivative of the signal might provide a better filter or allow for impulse detection. Code: input='MYSIN1.wav' %define input file output='test.wav' %define output file sample_size=256 %define fft size fs=44100 %define wave sampling rate noisef=7000 %lowest frequency to apply threasholding to tfreqs=700 %assumed range start tfreqf=2500 %assumed range end plotme=1 %enable plots for debugging samples_to_avg=8 %size of median filter ndebug=100; %debugging value for breakpoint every ndebug passes noisei=floor(noisef/fs*sample_size); %fft index tfreqis=floor(tfreqs/fs*sample_size); tfreqif=floor(tfreqf/fs*sample_size); counter=0; my_window=transpose(hanning(sample_size)); %define window [infile,msg] = fopen (input, 'rb', 'l'); %open input file if (infile == -1), error (msg), end my_temp=fread(infile,44, 'short'); %get wave header info outfile=fopen(output,'w','l'); %open file for output fwrite(outfile,my_temp,'short'); %write header info buffer1=zeros(1,samples_to_avg); %initialize buffers buffer2=zeros(1,sample_size/2); data = transpose(fread (infile, sample_size, 'short')); while(1~=feof(infile)) %repeat until end of input file my_temp=[buffer1 data]; %initialize for median filter my_new2=[]; for my_index = 1:sample_size %median filter my_median=median(my_temp(my_index:my_index+samples_to_avg)); error=my_temp(my_index:my_index+samples_to_avg)-my_median; my_error_median=median(error); my_new2=[my_new2 my_median+my_error_median]; end my_new2=my_new2.*my_window; %window output my_spec=fft(my_new2); %calc fft my_spec1=abs(my_spec).^2; %calc power spectrum my_min=min(my_spec1); %determine min value threashold=.02*(median(my_spec1(tfreqis:tfreqif))-my_min)+my_min; killedvals=0; for my_index = noisei:sample_size-noisei %spectral filter if(my_spec1(my_index)<threashold) my_spec(my_index)=0; killedvals=killedvals+1; end end my_new=real(ifft(my_spec)); if(plotme==1 && mod(counter,ndebug)==0) %plot output close all subplot(3,2,1); plot(my_temp); title('Input') subplot(3,2,2); plot(real(my_new2)); title('Median Output') subplot(3,2,3); plot(sqrt(my_spec1)); title('Input Spectrum') subplot(3,2,4); plot(abs(my_spec)); title('Clipped Spectrum') subplot(3,2,5); plot(my_new); title('Spectrum Output') threashold killedvals end fwrite(outfile,my_new(1:sample_size/2)+buffer2,'short'); %write output file buffer1 = data(sample_size-samples_to_avg:sample_size); %prepare data for next set buffer2 = my_new(sample_size/2+1:sample_size); data = [data(sample_size/2+1:sample_size) transpose(fread(infile, sample_size/2, 'short'))]; counter=counter+1; end fwrite(outfile,zeros(1,sample_size),'short'); %guarantee file length fclose(outfile); %close output file