Microphone Based Note Input Mode

Hello,

I always wanted a tracker that would be able to convert a whistle coming from a line-in device, into C-0 to B9 notes.
When I compose, I often hear the right note in my head, but I’m taking more time to find the corresponding note on the qwerty keyboard for example.
Renoise is one of the fastest way to compose music thanx to its intuitive interface. And it could produce even faster results.

In a near future, I would be happy to see that kind of feature in Renoise : a modulation meta device that converts a monophonic line-in signal into C0 to B9 notes and then would be able to route those notes to (1) VSTis, (2) the renoise note input buffer (usefull in Edit mode). It could also Modulate other controllers of FX like the keytracker already does for example.

Would it require a lot of new code ? Probably no.

When I get a look in the spectrum, I realize that Renoise already knwows how to precisely analyse the output signal and all frequencies between 20Hz and 20000Hz. If Renoise didn’t knew how to do it, it could not display such a detailed graph in realtime !

I also know that you can convert keyboard notes, into defined frequencies, (playable by the Ringmod device), thanx to last dblue efforts.
I mean that a monophonic incoming signal could be simply seen as an “average frequency” given a defined time and frequency range (2 parameters only) ; this average frequency could be simply converted into a note.

But when sounds you hear are “complex” and composed of fundamental frequencies and harmonics, you can detect precisely notes by using the following harmonics table as a valid reference :

C0,16.35,32.7,49.05,65.41,81.76
C0#,17.32,34.65,51.97,69.3,86.62
D0,18.35,36.71,55.06,73.42,91.77
D0#,19.45,38.89,58.34,77.78,97.23
E0,20.6,41.2,61.81,82.41,103.01
F0,21.83,43.65,65.48,87.31,109.13
F0#,23.12,46.25,69.37,92.5,115.62
G0,24.5,49,73.5,98,122.5
G0#,25.96,51.91,77.87,103.83,129.78
A0,27.5,55,82.5,110,137.5
A0#,29.14,58.27,87.41,116.54,145.68
B0,30.87,61.74,92.6,123.47,154.34
C1,32.7,65.41,98.11,130.81,163.52
C1#,34.65,69.3,103.94,138.59,173.24
D1,36.71,73.42,110.12,146.83,183.54
D1#,38.89,77.78,116.67,155.56,194.45
E1,41.2,82.41,123.61,164.81,206.02
F1,43.65,87.31,130.96,174.61,218.27
F1#,46.25,92.5,138.75,185,231.25
G1,49,98,147,196,245
G1#,51.91,103.83,155.74,207.65,259.57
A1,55,110,165,220,275
A1#,58.27,116.54,174.81,233.08,291.35
B1,61.74,123.47,185.21,246.94,308.68
C2,65.41,130.81,196.22,261.63,327.03
C2#,69.3,138.59,207.89,277.18,346.48
D2,73.42,146.83,220.25,293.66,367.08
D2#,77.78,155.56,233.35,311.13,388.91
E2,82.41,164.81,247.22,329.63,412.03
F2,87.31,174.61,261.92,349.23,436.54
F2#,92.5,185,277.5,369.99,462.49
G2,98,196,294,392,489.99
G2#,103.83,207.65,311.48,415.3,519.13
A2,110,220,330,440,550
A2#,116.54,233.08,349.62,466.16,582.7
B2,123.47,246.94,370.41,493.88,617.35
C3,130.81,261.63,392.44,523.25,654.06
C3#,138.59,277.18,415.77,554.37,692.96
D3,146.83,293.66,440.5,587.33,734.16
D3#,155.56,311.13,466.69,622.25,777.82
E3,164.81,329.63,494.44,659.26,824.07
F3,174.61,349.23,523.84,698.46,873.07
F3#,185,369.99,554.99,739.99,924.99
G3,196,392,587.99,783.99,979.99
G3#,207.65,415.3,622.96,830.61,1038.26
A3,220,440,660,880,1100
A3#,233.08,466.16,699.25,932.33,1165.41
B3,246.94,493.88,740.82,987.77,1234.71
C4,261.63,523.25,784.88,1046.5,1308.13
C4#,277.18,554.37,831.55,1108.73,1385.91
D4,293.66,587.33,880.99,1174.66,1468.32
D4#,311.13,622.25,933.38,1244.51,1555.63
E4,329.63,659.26,988.88,1318.51,1648.14
F4,349.23,698.46,1047.68,1396.91,1746.14
F4#,369.99,739.99,1109.98,1479.98,1849.97
G4,392,783.99,1175.99,1567.98,1959.98
G4#,415.3,830.61,1245.91,1661.22,2076.52
A4,440,880,1320,1760,2200
A4#,466.16,932.33,1398.49,1864.66,2330.82
B4,493.88,987.77,1481.65,1975.53,2469.42
C5,523.25,1046.5,1569.75,2093,2616.26
C5#,554.37,1108.73,1663.1,2217.46,2771.83
D5,587.33,1174.66,1761.99,2349.32,2936.65
D5#,622.25,1244.51,1866.76,2489.02,3111.27
E5,659.26,1318.51,1977.77,2637.02,3296.28
F5,698.46,1396.91,2095.37,2793.83,3492.28
F5#,739.99,1479.98,2219.97,2959.96,3699.94
G5,783.99,1567.98,2351.97,3135.96,3919.95
G5#,830.61,1661.22,2491.83,3322.44,4153.05
A5,880,1760,2640,3520,4400
A5#,932.33,1864.65,2796.98,3729.31,4661.64
B5,987.77,1975.53,2963.3,3951.07,4938.83
C6,1046.5,2093,3139.51,4186.01,5232.51
C6#,1108.73,2217.46,3326.19,4434.92,5543.65
D6,1174.66,2349.32,3523.98,4698.64,5873.29
D6#,1244.51,2489.02,3733.52,4978.03,6222.54
E6,1318.51,2637.02,3955.53,5274.04,6592.55
F6,1396.91,2793.83,4190.74,5587.65,6984.56
F6#,1479.98,2959.96,4439.93,5919.91,7399.89
G6,1567.98,3135.96,4703.94,6271.93,7839.91
G6#,1661.22,3322.44,4983.66,6644.87,8306.09
A6,1760,3520,5280,7040,8800
A6#,1864.65,3729.31,5593.96,7458.62,9323.27
B6,1975.53,3951.07,5926.6,7902.13,9877.66
C7,2093,4186.01,6279.01,8372.02,10465.02
C7#,2217.46,4434.92,6652.38,8869.84,11087.3
D7,2349.32,4698.64,7047.95,9397.27,11746.59
D7#,2489.02,4978.03,7467.05,9956.06,12445.08
E7,2637.02,5274.04,7911.06,10548.08,13185.1
F7,2793.83,5587.65,8381.48,11175.3,13969.13
F7#,2959.95,5919.91,8879.86,11839.82,14799.77
G7,3135.96,6271.93,9407.89,12543.85,15679.82
G7#,3322.44,6644.87,9967.31,13289.75,16612.19
A7,3520,7040,10560,14080,17600
A7#,3729.31,7458.62,11187.93,14917.24,18646.55
B7,3951.07,7902.13,11853.2,15804.26,19755.33
C8,4186.01,8372.02,12558.03,16744.03,20930.04
C8#,4434.92,8869.84,13304.76,17739.69,22174.61
D8,4698.64,9397.27,14095.91,18794.54,23493.18
D8#,4978.03,9956.06,14934.09,19912.12,24890.15
E8,5274.04,10548.08,15822.12,21096.16,26370.2
F8,5587.65,11175.3,16762.95,22350.6,27938.25
F8#,5919.91,11839.82,17759.73,23679.64,29599.55
G8,6271.93,12543.85,18815.78,25087.7,31359.63
G8#,6644.87,13289.75,19934.62,26579.5,33224.37
A8,7040,14080,21120,28159.99,35199.99
A8#,7458.62,14917.24,22375.86,29834.47,37293.09
B8,7902.13,15804.26,23706.39,31608.52,39510.66
C9,8372.02,16744.03,25116.05,33488.07,41860.08
C9#,8869.84,17739.68,26609.53,35479.37,44349.21
D9,9397.27,18794.54,28191.81,37589.08,46986.35
D9#,9956.06,19912.12,29868.18,39824.25,49780.31
E9,10548.08,21096.16,31644.24,42192.32,52740.4
F9,11175.3,22350.6,33525.9,44701.2,55876.5
F9#,11839.82,23679.64,35519.46,47359.27,59199.09
G9,12543.85,25087.7,37631.55,50175.4,62719.25
G9#,13289.75,26579.49,39869.24,53158.99,66448.74
A9,14080,28159.99,42239.99,56319.99,70399.98
A9#,14917.24,29834.47,44751.71,59668.95,74586.18
B9,15804.26,31608.52,47412.78,63217.05,79021.31

I don’t ask of course something that detects polyphonic chords. No. Just something able to guess a simple note, from a monophonic line-in signal; and if it could be routed to VSTis or if it could modulate something else with it, that would just be perfect. I Maybe I’m wrong, but I guess that the main code and logic of this input system is already there in Renoise, and that it has just to be re-used in another way.

seeya

yes yes yes. i’ve tried this with other tools… but it’s very hard to make it work in real time… (doesn’t really have to be realtime perhaps idk) and i used real time pitch correction to get pure notes… before they would hit the vst synth. it’s a fun way to work with for sure, very creative. one is probably better of getting a hardware to do the audio-2-midi… but who knows, i love to have it in software form!

If that is really kinda easy to implement, that would be bloody great!!!

+1

Isn’t this how, for example, guitar tuners work? They listen to the frequency and converts it to a note, right?

+1

yep, something like that already exist and for free,
a tuner is also a ‘pitch monitor’
check for example the following URL : http://tuneit.free.fr/Site/Home.html
but in this example, the pitch monitoring is focused around the subtle variations around the American Standard Pitch value of A4 = 440 Hz (as default).

In my mind, that kind of ‘renoise pitch monitor’ should be able to check a range of 20 to 20000 Hz. It would not only monitor the C0-B9 notes, but those monitored notes could be automatically re-used or routed to the input keyboard buffer. For example, you could whistle, and the monitored notes would be inserted in realtime, in the pattern editor while in edit mode. You could also route discovered notes to VSTi instruments. You could then whistle and hear in realtime the playback of another VST instrument. The device could in the end act like the keytrack device, using monitored notes, to modulate something else (DSP, VST effects, LFO, what you want). The ability for this new hybrid device (because it is a line-in + route + modulation device ) to do everything in realtime should definitively become one of the most killer feature of Renoise.

Expensive (USD 249)… but doable:
http://www.celemony.com/userforum/viewtopic.php?t=4647

On the upside, it is able to pick up every intricate little pitchbend and yet still snap the main frequencies to notes, which could make for some interestingly ‘natural’ leads.

It’s actually quite complex to track the pitch. The complete spectrum is exactly that, the complete spectrum. It doesn’t tell us about the pitch, but rather is the sum of everything. Many mathematical models exist for extracting the perceived pitch, for example the “Meddis Unitary model”

http://www.essex.ac.uk/psychology/psy/people/meddis/Meddis_OMard%201997(Pitch).pdf

If someone finds it useful, I’ve experimented with pitch-tracking using CPS.
CPS is a freeware modular environment with pitch-tracking capabilities.
http://cps.bonneville.nl/fullversion.php

Hint: the pitchtracking module is called “Apollo”

Did you tried Akoff Music composer?
It translates mic or wav input to MIDI notes which in turn you can send to Renoise using Midi Yoke or another virtual cable.
It will always be hard to make it work in realtime, also due to alias effects that might be measured.
The only thing that can make this work the best is a good mic and a very good noisefilter.

danoise you’re quite right. A human voice that sings, for example, produces a lot of variable spectral informations. That’s why I asked for something that would be able to convert a “whistle” into a note.

Because I’ve noticed that a “whistle” is easier to analyse than complex vocal signals. For example you put a line-in device, then you filter the signal with a high pass (0.18Kh, butterworth 4n model) and you put a gater. If your microphone isn’t amplified, add a gainer. Then whistle in your microphone, click on the renoise spectrum view. You won’t see anything hard to track at all : you’ll see a big simple peak. While I’m writing this line, I’m checking my spectrum, I can see fundamentals values from 1K to 2K depending on the note I’m doing with my mouth, and then I can see 4 very small peacks on the right side, probably corresponding to harmonics.

That’s a child’s play.

Is there anything in that kind of free opensource code that the dev-team isn’t able to understand because it’s too complex ?:
http://ling.upenn.edu/~kgorman/c/swipe/
or
http://sourceforge.net/projects/otuner/files/
No.

You’ve got also a lot of source code inspiration in free audio libraries like :
http://sourceforge.net/projects/c-four/files/


That one is a roll’s royce :lol:. Melodyne is not only a powerfull pitch recognition software (it converts every kind of note to midi notes, it’s able to automatically write scores …, and it detects even chords!!!), it’s also a tool that precisely correct tones with a simple “drag and drop” system. I don’t expect that much !

I’m just focusing my mind on pitch detection, and most of all, the integration of a detected note in the renoise global logic, (route the monitored note to a VSTi, modulate other dsps or VST effects, or feed the input keyboard buffer with it)…

I’ve used one of those realtime pitch to midi converters with a bass guitar.
It was interesting using it with reaktor. There was a bit of latency that needed to be compensated for.
For instance by the time reaktor would get note 1, I would already be playing note 2.

answer = http://sourceforge.n…ts/ultrastardx/
(open source clone)

Bidule has a spectral to midi converter too. I’ve used it to record my theremin into Renoise in the form of midi notes. Worked great. Bidule is also free to use while in beta, though I own it.