continue releasing in small steps

General Discussion
1

So far we had lots of small releases with many small improvements, and I’m no longer sure if this is the way to go.
The smaller the releases, the higher is the probability that we have intoduced bugs that none of the betatester has found (as the beta periodes are quite short). On the other hand : The bigger the release is, the higher is the probability that we’ve introduced more bugs that have to be found.

Most big commertial companies do releases in two steps : One big 1 versionnumber step to get again some money from the customers and one half version number release with small improvements wich are usually free. I dont like to have this for Renoise as I would like to let you benefit from the small changes we fix and add inbetween.

We also have to struggle with lots of code that I’ve written in my early days as a programmer wich definitly needs to be changed or fixed, to be ready for the big changes that we’ve planned so far (for example the mac port, pianoroll or zoomable patterneditor). So, many (also existing) code will be changed and we have to be very carefully to not indroduce more bugs.

What do you think is the best way for Renoise ? Keeping the 0.1 version updates with small improvements or having even smaller or bigger releasesteps ? What could we do to make the releases more stable ? Having a weekly updated build in the userpages, or as it is now only one or two weeks before we will release ?

2

I think milestones and weekly builds will be the best way (in the way mozilla and phoenix is developed). Then the people that think it is okay with alittle unstable releases can go for the weekly builds and can in this way also help with the beta-testing of the real milestone (released every second or third month or so)… But if you introduce the weekly buids method it realy has to be possible to download more than just the latest build, casue else you will be screwed untill the next build if a critical bug shows up…

In this way you get continuous beta-testing and more stable milestones…

3

After handling tunes with evolving tools through the years… one learns one weird concept that might sound funny:
“boundaries can make it easy for you to be creative”

When a small number of options is available your creativity allow you to “play” on those boundaries using them as source for inspiration.
All in all making tunes for a digital wristwatch is far easier than composing on Renoise, if you see what I mean…

I’m thinking about a way to apply this concept on this problem as well… and it turns out that infact you renoise developers have certain boundaries… If my theory is true, we should find inspiration to know how to proceed by looking at what you really can’t afford to run into.

Renoise is a commercial program to make music… so basicly you can’t afford to Release a badly non-working program because that’s what people has come for. The free use of Renoise demo, supposed to get more users to register, could instead turn them away from it.

Also, Renoise is a concept.
When you advertise, you build an idea into people. It’s not a random accident that you advertise about Renoise’s features! Infact you should be aware that you build “the idea of a function” by promising the function to be available to the user. You can’t afford to be swollen… and a lot of continous patches and upgrades and changes are proved to result in a suggestion of “non-reliable” more than “accurate bug-fixing”

Well… what do we end up with?

We end up with the actual situation.
I will assume that we can speak about some Renoise 1.27 as a starting point, as a base for future improvements.
Have this 1.27 perfectly working… next step could be to gather planned features and implement them into some X beta exactly as you would normally do.

Distribute the beta by warning it’s a beta… and leaving the perfectly-working 1.27 as main download… possibly in some way that makes it clear to people that 1.27 is the main, complete, full, extracool, professional version… and "if they really really want to betatest they can download the Xperimenta-Hybrid-incomplete-bugged betarelease.

Obviously every beta should challenge with some new really-exiting improvements… so I also suggest to slow down a little bit with the releases in order to have more consistent changes to every update. This will create slightly longer betatesting periods but you assure Renoise perfect final releases and interesting new releases that means happy costumers and a solid image.
:)

4

fast response to bugreports is a must.

we cant wait weeks for bugs to be fixed just because the fixes are in the next big version which wont be released until all the new options for that version are finished too.
(I think that was the case with 1.23 to 1.25 (if I am right))

5

my thought about it:

about releasing and testing:

  1. release stable version only, stable as possible. (btw, i’m pretty happy with the current release. stable as hell if compare it to any of 1.00-1.25)

  2. NEVER release anything without the beta testing.

  3. give 48 hours to beta testers (only) to check FINAL release before publishing.

  4. give more detailed build numbers for the beta testers (ABCD, AB=absolute number of the build, C=number of added features since previous build, D=number of fixed bugs since previous build)

  5. make a forum for beta testers with categories: “Found bugs”, “Fixed bugs”, “Added features, pay attention”, where each topic starts with the build number.

  6. make a test song for the beta testers, with using of ALL available features. the song starts with base command checking with all of kind of combinations, then effects and automation. then it could be rendered on latest build and checked digitally with already tested one.

about developing:

  1. calculate potential developing power and give different parts of it to gui/engine developing. for example 30% for gui, 50% for engine, 10% for optimization, 10% for bugs fixing. then use it according to user voting.

  2. use mixed scheme of version steping (for example +0.05, +0.1, +0.05, +0.2) and synchronize it with todo list.

  3. put “hard to code” features to background developing.

  4. create “designers team” where people (designers) will optimize ideas and create new optimal standarts for new features.

  5. make a forum for the designers team.

6

If you choose the way to work I suggested the final release shall realy be beta tested more than 48 hours. And the final release candidate shall be identical (the same build) as the final release.
The weekly builds shall be continued untlil all known critical, major and normal bugs are fixed, and untill no new bugs are found, only minor and trivial bugs (like: “the reder-bar is one pixel too small” or something similar ;)) can be left in milesstones. And as more betatester as higher chance all the bugs are found, therefore the weakly builds shall be available to to the public (at least all the registerd users). Only completly new versions (“alfa releases” not “binary compatible” with the previous one) shall be tested by beta-testers only.
Also establish a better bug-reporing system, the way it works today it doesn’t seems very organized and therefore probably lot of bugs are missed… the bug-reporting system shall also have a public list with all reported bugs listed (numbered and rated, eg #432. The Play Button does not work. Rated: Critical), in this way persons can watch if the bug they are going to report is already reported. It would also be great if this list is threased so it is possible for other people to put commets and updates to old bug-reports.

7

a “stable/experimental” build system would be great, and infact is a widely used system for all the fast evolving programs out there.

by the way, as a quite involved betatester, I already have at least 2 test builds between versions on my HD, so I don’t care if you won’t do it :veryevilgrin: :D

8

ok, what about this then :
we will have more frequently beta builds in the userpages (maybe every 2 weeks or 3 a new update.) They are always marked as betas, so if you want to help us, you use the beta, if you have to work on some serius stuff you use the lasted stable taged version.
we also need a better bugtracking system then. I took a look of all bugtrackers that I could find but every of them sucked. Either it was far too complicated or it simply didnt worked at all. As we have a smart guy call pulsar in our team :) , who loves to do web programming, we might ask him to write a small bugtracker for us that fits our needs.
Pulsar : what do you think ?
I thought of having a simple sortable list with the current entries :

Author : Date : Description : (hideable long description *) : Version : State : responseable developer

  • with room for questions from the developer to the author of this bug.

new entries can be made only by registerd users. the state and responsable developer entri can only be edited by developers …

9

what I forgot : maybe the bugtracker could even be integrated into the userpages ?

10

Have you looked at bugzilla?

11

taktik: already on it. if ITtracker (my hopes are with this one), doesnt work for us, i will write a small renoise-only bugtracker.

IF i am going to write one, i’ll open a new thread about feature-request and those priorities. as always: most tricky part first, then the needed features and last but not least the nice-to-have ones.

a simple bugtracker should not take longer than a weekend to code. if we are not bound to renoise.com (like the forum is actually in a frame with another origin), it will be servlet based, since there are already plenty of free libraries available - http://jakarta.apache.org/

regarding bugzilla : are you crazy ? ist a very good bugtracking system for skilled developers and big projects, but for a single one, with very few “user-visible-components” its just an overkill.

12

i think a non public forum for beta testers could be more effective than a bugtracker. i check my user page only to download renoise. it’s pain to go to where you need to login first and then do something. in my opinion a connection between people is the power.