I have had this problem myself, found one or two VSTs which didn’t really work.
What works quite well for me is starting Reaper as a reWire slave. You can import quicktime files there. Haven’t tested it very thoroughly yet, only with short clips. Most of them worked, one not so much. Guess it’s a matter of picking the right codec/fps/resolution or whatever. So some rerendering or cutting into smaller pieces might be necessary.
Processing can play assorted video formats as well as animated GIFs. In my article I have some tracks in Renoise dedicated to sending MIDI messages to a Processing sketch that changes what it is playing based on the note sent. I also used a Launchpad to do ad-hoc video updates.
You may also make an extra track in your renoise song, insert sounds exactly where you want to trigger events in your video, then render the track and convert into keyframes using After Effects. There are lots of possibilities with this, you can turn volume fades into resizing video objects or similar stuff very easily.
Hey celsius; I’m wondering if a VJ app would be a solution for you.
I use Renoise envelopes to control the play head, play speed, loop start and loop length positions of a couple different Mac VJ apps ( ImXIntel - http://image-ine.org/ and CoGe - http://cogevj.hu ), and i know there are many other VJ apps that will work with Renoise as well.
It’s doable because Renoise can send MIDI CC values out to any other app that can be controlled by MIDI (like many VJ apps).
Given your mention of Fruity (FL Studio?) I’m guessing you’re likely using MS Windows, but you have multiple options there too.
vvvv ( http://vvvv.org ) is one of my favourite Windows apps; it may be over-kill for your purposes, but it’s free for non-commercial use and could suit your needs.
Course, there are many other VJ apps for Windows as well.
I don’t mean this to be a knock on Renoise, but its amazing this is still an optional capability for a DAW in 2014! (Renoise isn’t the only one) I mean, I was scoring to reference video (QT) within Studiovision Pro by 1997. (before that I was doing SMPTE with a 3/4" Umatic deck) One would think at least basic, synchronized playback of one or two major video formats (even just thumbnail reference video) would be de rigeur for any DAW by now. I wonder what this looks from the development side? Maybe there’s a reason such a thing isn’t in Renoise (yet)?
this is still a huuge potential market niche that nobody has bothered to explore yet, for reasons unknown to me.
I remember Coldcut of NinjaTune fame once had a software like this in the making, back in the late 90-s. They never delivered tho. https://www.youtube.com/watch?v=vBH75VrSBxs
But this is just for the first case, where you can have a video track that you can scrub and step through while arranging and sequencing music to go along with it.
I’ve thought about how cool it would be, to be able to 0SXX around a video clip inside a tracker like renoise, for a long time. It would be boat loads of fun, and you could do some amazing stuff.
For a minute I thought about sampling Max/MSP’s Ms. Pinky records and trying to just cut those up in Renoise and then feed them back into max. Never got around to doing it though.
In the ideal case you’d have to basically swap out the whole audio decoder, and then put in a video decoder engine, and have your sample editor behave like a mini video editor (sort of like the video editor in the iPhone). I think it would be obnoxious to try to maintain on three different platforms since Windows, OSX and Linux all have their different notions for how to do video. Then, you’d also need to probably use uncompressed video, since most lightweight playback codecs don’t do random frame addressing and reverse playback without additional I/P-frame lookahead, so yeah… it’s a really fun idea, but massively complicated to implement.
There’s also the fact that in the last few years I’ve gotten so happy about playback performance using modern computer hardware. I don’t remember the last time I crashed a DAW, whereas back in the 90s and early 2000s it was like a constant threat that duplicating this track, or doing this edit during playback would make the whole system die and loose you hours of caffeine induced tweaking. It would probably require some hard core gear to run a multi-track video tracker that can do crazy sample chopping and retrigger on more than one or two video samples at once.