Planet MB

I’m pleased to announce that the MetaBrainz Foundation has hired its first full time developer! On Februrary 1, 2010, Kuno Woudt will join Oliver Charles on the MusicBrainz development team and both of them will focus to finish NGS as soon as possible. With Kuno we are effectively tripling the engineering capacity we have dedicated to writing code for NGS. This will free me up to focus more on managing the development process as well as focusing on more business development issues. An increased focus on business should allow me to find the extra income to pay Kuno and to keep us in the black!

I’m excited by this development — having dedicated developers working on finishing NGS is what has been needed for several years now. I’m very much looking forward to releasing NGS and then having a number of smaller releases in 2010!

Navap undertook a re-write of our privacy policy page to make the page more consistent and better organized. However, the actual policy did not change — just the wording/formatting of the page has changed.

Please take a look at the diff of the changes and the new page.

(This post is mostly only for full disclosure — seeing Facebook being shredded for its new policy is making me overly cautious right now.)

We’ve just finished our 10th MusicBrainz Summit, this time held in Nürnberg, Germany. It was a great success, with a number of participants and a lot of discussion! This time, we’ve managed to note all the discussions. So, if you missed the summit and want to catch up on what we talked about, head over to our wiki page with the summit notes, which contains a summary of just about everything we mentioned.

Thanks to everyone who came, and bring on the next summit!

Picard 0.12.1 is an important hotfix release. Due to a bug Picard 0.12 deletes all ID3 comment tags (COMM frames) when saving files. In addition this release restores the native file dialog when adding folders on Windows.

Picard 0.12.1 is available for download for Windows and Linux.

Sorry for the trouble.

The VMWare image has been updated for the 2009-05-24 server release. The previous image was getting old and updating steps have been reported not easy.

The new image has been upgraded to Debian 5.0, includes a more recent Linux kernel and uses PosgreSQL 8.3 (the required version to work on NGS).

To download and play with the new image, read our VirtualMusicBrainzServer wiki page!

The fabulous Music Hack Day is happening in Boston on November 21-22. Sadly, I will be in London/Hamburg that weekend and can’t make it to Boston. The organizers have requested that we have someone represent MusicBrainz at the hack day. Are there any MusicBrainzers who live in/near Boston who would like to go to the hack day and represent MusicBrainz?

Knowledge of the project, our web interface, MBIDs and our web service would be really useful to have. Please post a comment if you’re interested. And even if you’re not interested in representing MB, it looks to be a great event that you should go to in any case!

UPDATE: Alastair Porter has volunteered! Thanks Alastair!

We have released the next version of MusicBrainz Picard. Picard 0.12 includes a lot of bug fixes and new features, including:

• Support for ratings and folksonomy tags.
• Live syntax checking for tagger script and naming strings.
• Embed cover art into WMA and APEv2 tags.
• New script functions $matchedtracks(), $initials(), $firstalphachar(), $truncate() and $firstwords()
• New plugin extension point ui_init, allowing plugins to add new UI elements to the main window.
• A new high quality application icon.
• Support for originaldate tag. While this is not filled by Picard itself it can be used from within plugins such as the Original Release Date plugin.
• Write ISRCs from MusicBrainz into tags.
• CD drive dropdown selection on Linux.
• Various small improvements to the UI.
• Updated translations and the option to choose the user interface language.

A complete list of changes be found in NEWS.txt.

Picard 0.12 is available for download for Windows and Linux. The Mac OS X version will be released later, sorry for that. We are still in search for a long term maintainer of Picard on OS X.

Thanks to everybody who contributed to this release.

I’m about to finish writing the web service (version 1 & 2) for NGS, based on the work that Lukas started a few months ago. The NGS web service currently does not have a compete means of tweaking how much XML is returned for a given resource. Lukas wondered if we should keep our existing approach or start using a new approach that Last.fm (and others) use.

Compare our current approach:

/ws/1/release-group/052c7adb-5e2d-3cf3-b303-6c2a7d3e5b1c/?type=xml&inc=releases (docs)

to the Last.fm approach:

/2.0/?method=album.getinfo&artist=Cher&album=Believe (docs)

Should we keep our current resource focused URL structure or move to a method centric approach in our v2?

If you’re a fan of the old skool structure, please leave a comment and tell us what could be improved. What do you like about the web service? What do you hate?

For some background — as of the NGS release we will be making a lot of changes:

• the old RDF web service will no longer work.
• The old v1 XML interface will continue to work, but not optimally since we will be shoe-horning the spiffy new NGS XML into the old skool v1 XML. Concepts like artist-credits and works will not be available in the v1 compatibility interface.
• The new v2 XML interface will expose all the NGS goodies like artist-credits and works.

[ MusicBrainz readers: Please ignore this post -- its only so that others can find the solution to a problem I just solved ]

If you are running the Perl module Text::Unaccent (version 1.07) on a 64 bit system, you’ll find that it fails its test cases. This is due to a mismatched type in the Perl module. Apply this patch to your Text::Unaccent code, recompile and the tests will pass!

When our first beta testing date rolled around we were not quite ready for testing and had hoped that in two weeks we could catch up and be ready. Sadly, that was not the case. We’ve made some changes in our team and are redoubling our efforts to get a working release editor finished as soon as possible.

Since we’ve made some changes our pace going forward is not clear yet, which makes setting a new schedule very hard. We’re now ready to start using our bug tracker to keep track of remaining tasks and bugs as they appear. To see how we are progressing on finishing NGS, take a look at the MusicBrainz NGS (Beta 1) milestone.

I hope to have a more firm schedule soon. Stay tuned.

We’re going to be changing how the traffic routes through our servers on Sunday September 13th near mid-day PDT, late evening UK time. In theory you should not notice any traffic interruptions, but theory doesn’t always match up with practice.

Before we start the maintenance, we’re going to announce the switch over to the new route in our IRC channel #musicbrainz on irc.freenode.net .

With a little luck no one will notice our tinkering.

Update by djce: The change is now complete and seems to be working just fine.  Thank you for your patience.

Paul Taylor and Aurelién Mino have completed the re-write of our search server — please help us test this new code-base. Here is Paul’s write up:

This update rewrites the existing Search Indexer and Searcher with a
pure Java release whilst maintaining full compatibility with the
existing Musicbrainz Server, prototype by luks, completion by ijabz and
murdos. The aims were

1. Better performance

Search performance should be greatly improved for indexing and
especially search.

The intermediate python layer has been removed.
The webserver layer has been removed
When searching have a single Java VM , that can now be properly
configured/optimized
The latest Lucene and Java libraries are used

2. Better Search Results

The following bugs have already been fixed

Perhaps most importantly stop words are no longer and issue so when do a
search for something that exists in the database you should always get a
match, i.e artist:”the the” or track:”is this it?”

example

and range queries work properly

i.e

example

and theres a host of other bugs and enhancements that can now be
resolved quite easily

bug list

3. Maintainability

The code has been simplified, i.e no hand coding of xml or html or
having to deal with escape characters.
Now has a full set of unit tests, can also easily generate metrics such
as code coverage reports.
No dependency on Linux or Python, making it easier for new developers
to contribute, removing the burden from ruaok.

Testing Stage

Please try it out on http://musicbrainz.homeip.net both as a websearch
and as an XML Service to find any omissions in functionality and any
situations where the search results aren’t good as before.

Paul / ijabz

I’m sad to say that we’re a little behind on getting our core features finished for NGS. Given that, we’re pushing back the schedule to start NGS testing on September 15th. We need more time to finish working on the complex release editor and round out a number of other features.

Stay tuned for more details!

Mark your calendars: The next MusicBrainz summit will happen on 28 November, 2009 in Nürnberg, Germany! More details will come in the following weeks…

Use frozen carbon dioxide (dry ice), mix it with water and pipe the steam into a PET bottle. When the bottle at roughly 22bar explodes you can shoot for example a toy chicken over more than 80 meters (We had serious problems to build a projectile that is not destroyed by the power of the launch). We built the device for a creativity competition at our university.
And well, we could not place an object accurate on 20m but for shure we did it with the biggest impact.


Music on Amazon and iTunes

If you are curious, here some detail about the construction:

For us secure handling was not only an option. We have seen some people that put water and dry-ice together into an bottle close it and put it into a pipe. But you have good chances that it will explode in your hands. So we put an PET bottle into our pipe and connect it with an CNC made adapter to the presure reservoir outside. Beside that, all parts are made from steel and welded. We experienced that a 1.5l coke bottle (actually we used Deit bottles that explode in a better way) can withstand up to 22bar of pressure (what is more than most sources on the internet report)

The next design decision we had to make was a system to quickly inject all the water while not blocking the exhaust of the considerable amout of CO2 gas that is generated as the water mixes with the dry-ice. We first tried it with an single valve but had a problem to get more water it at some point when to much CO2 was exhausted. Howerver two, one for water injection and the other valve for the gas exhaust were ways better.

Finally, the manometer is not necessary but nice to have as it makes the firing a bit less surprising. All in all we spent about 90 Euros.

As we’re working hard on the next generation schema, (NGS) we’re now deep in the middle of working on the release editor. We’re working hard to embrace modern UI design concepts and we’re shooting for a clean and uncluttered look. Part of this effort will require the creation of a number of icons that fit in our new site design.

Does anyone know of an icon designer who would be willing to help us design a handful of icons? This is an important aspect of our next release, so if we need to pay for professional icon design, so be it. We would of course love to find a qualified volunteer or perhaps a designer who would be willing to give us a discount.

If you know someone, please leave a comment.

After many years of planning, anticipating and gathering resources, we’re finally tangibly close to our Next Generation Schema (NGS).

This blog post is intended as an official notification to everyone in our community and our customers who are using MusicBrainz services right now. Our next generation schema is drastic evolution for MusicBrainz — we’ve included many of the features that our customers and our community has asked for over the past few years. This means that if you’re using the MusicBrainz data right now, you will need to prepare your systems in order to be ready for the switchover when it comes in the fall. Please do not delay examining our new schema — this change is drastic change from our previous schema!

Our current plans are to enter beta testing of the new server on August 31. The exact release date is very much dependent on the results of our beta phase, but I hope to have the release within 60 days of entering beta.

The most important changes that you will need to consider/address:

• No old (RDF) web service — the old RDF web service will no longer be supported as of NGS.
• We will provide an XML v1 web service that is backward compatible to our current XML web service.
• We will also provide an XML v2 web service that will expose new NGS concepts.
• Postgres 8.3 will be required. Upgrading your old database will not be possible. You will be required to import the first post NGS data-dump in order to upgrade to NGS. Our provided upgrade script (see below) is very useful for testing purposes but not suited for upgrading deployed servers.
• MBID changes — The MBIDs will be stable and maintained for artists, release-groups and tracks. All of the MBIDs for our current releases will also be kept, but we are changing what we are calling releases. Essentially all release events (with label, date, country and barcode information) will become releases each with their own MBID. This means that we’re adding a whole slew of new MBIDs for the releases that will not be assigned a legacy MBID.

NGS is documented on our wiki — please take some time to read up on our documentation! If you’d like to play with NGS now, follow these steps:

1. Download and install the 20090524 release according to these instructions. For going to NGS, installing a database only install is the perfect approach. Download and import an existing data set.

2. Download the NGS codebase with subversion.

3. Follow the install instructions. Instructions are included for how to migrate the 20090524 data to NGS — please note that the upgrade script may run for quite some time!

There are a lot of changes to the database from the current release! Please note that we’re done with the overall database design — I am not anticipating major changes past this point. However, I do anticipate a few smaller changes as we get closer to our goal. We will keep the schema diagram and documentation up to date with our changes.

If you care to follow our progress getting to NGS, please see our roadmap.

Google’s Open Source Office has once again decided to support MusicBrainz with another $30,000 donation! Read about how we use Google’s money on the Google open source office blog:

Last year, Google’s generous donation paid for a much needed server and it allowed us to hire our Google Summer of Code™ student (Oliver Charles) part time after the program wrapped up. The donation also helped pay for mundane things like keeping the lights on, backup disks and paying for insurance. But the most fun part that we spent money on last year was our phenomenal MusicBrainz Summit in London.

Thank you very much to everyone at the Google Open Source Office! MusicBrainz would be moving slower and be much more dull without your support!

The WSJ just posted a well balanced review of TuneUp and Picard.

Thanks for the nice write up, Geoff!

Matthias Friedrich (yalaforge) finds himself with little spare time on his hands these days and has asked me to find a new maintainer for the Python MusicBrainz library (pymusicbrainz2) and for the C based libdiscid library. While libdiscid doesn’t require immediate work, the pymb2 library needs to have support for Release Groups (from the 2009-05-24 server update) added in.

Matthias suggested that anyone interested in becoming the maintainer should write a patch to add the needed release groups support. I think this is a good idea — anyone interested?

If so, please post a comment!

And thanks for your hard work on these projects Matthias!

UPDATE: Our Help Wanted page has been updated to reflect our current needs.

While I did some research on how getting this to work I saw many people trying this without success. Here is the definitive guide on how you use OpenDNS with your Fritzbox:

1. Make sure your firmware is up-to-date
2. Enable telnet
   
3. Login over telnet and type "cd /var/flash/", "nvi ar7.cfg"
4. Search for overwritedns1 and overwritedns2 (they appear two times each) using the "/" key and "n" (next search result).
   
5. Press "i" to get into edit mode and enter the opendns server as noted in the opendns linux guide
6. ESC brings you back to command mode. ":w" saves the file and ":q" closes the editor again.
7. type "reboot" to restart the fritzbox
8. Restart you networking and you browser. The OpenDNS Site now should tell you that you are using OpenDNS already. Now you can create an account, set it to dynamic DNS, add network and login to dnsomatic.
9. Now login to fritz.box using your browser and enter the following dynamic DNS settings: Update URL: updates.dnsomatic.com/nic/update?myip=<ipaddr>
   (This is the URL used by your fritzbox to send the new ip address to DNS-O-Matic)
   Domain name: Nothing here.
   User name: this should be your username.
   Password : your password
   password confirmation: you password again.

Now everything should work fine.

A quick FYI before I go back to heads down mode for NGS — last night I imported the CD Baby metadata into our CD Stub collection:

Imported 196642 CDs. Already had 8382 CDs

We now have almost 50% more Disc Ids in our collection.

I just pushed a set of fixes to the main servers. To see what has changed, check out the recently closed bug list. More bug fixes will come later this week.

The old skool RDF based web service (at /cgi-bin/mq_2_1.pl /cgi-bin/mq.pl /cgi-bin/rdf_2_1.pl and /cgi-bin/rdf.pl) will cease to exist when we release the Next Generation Schema (NGS) release that will go into a beta release on August 31. This web service has been deprecated for three years now, its finally time to put it out if its own misery.

As of this release the Classic Tagger will completely stop functioning. RIP Classic Tagger!

We’ve just finished rolling out our latest release to support Release Groups, ISRCs and CD Stub searching!

For all the details on this release, check out the release notes.
I would like to thank Luks, Dave Evans, Navap, Outsidecontext, Voiceinsideyou, Murdos, Pronik, Prodoc and everyone else who has helped put this release together! Thank you for all your hard work!

P.S. Happy Birthday to Matt Wood!

Welcome all Lifehacker users!

Unfortunately you picked a bad day to come inundate MusicBrainz, because today is our next server update. This will start very soon, within about half an hour of me posting this entry. We expect to be down for at least 30 minutes today.

Sorry for the inconvenience!

Wolfram Alpha, the “computational knowledge engine” that has been the buzz the last few days uses MusicBrainz data! Click on the source information link on their example search for music:

When was Like a Rolling Stone released?

They use our data, but unfortunately they are not our customer.

We’ve hammered out many of the rough edges from the upcoming release and now we need your help to test the release to spot any problems that may have slipped past our new code review process.

New in this release:
- Release groups: This allows us to group same titled releases from one artist that have slightly different track lists into release groups. For instance, here is a Weezer release group that has many separate releases in it. We have converted as many batches of releases to release groups as we automatically could, but there are tons left to do. We’ll need your help!
- ISRC support: We can now track ISRC codes. While this is less useful to end users, our commercial customers have been asking for this for eons.
- WikiDocs: Our WikiDocs system now uses our new Mediawiki to pull documentation from.
- Bug fixes from our last release.

Aside from a good chunk of the bug fixes, all of these things are now live on our test server. Please report bugs to our bug tracker and make sure to select the “Server: ReleaseGroups, ISRCs, Bug Fixes” milestone so we can spot your bugs fast. Also take a look at the bugs we’ve already closed for this release and which ones are still outstanding.

We have one major known issue, where some release groups may be found in the search engine, but will give a “release group not found error” (example). This is a known bug.

Finally, do the release groups as we have them now make sense to you? There are a few things that may not be entirely clear, so we’re looking for feedback how to make things more clear before we release this on May 24th.

The next update of the MusicBrainz server will happen on May 24th. For this update we will have the following new features:

1. Release groups: Group together like releases (same titled releases by the same artist) into one group.
2. Support for attaching ISRC codes to tracks.
3. Bug fixes.

No later than this weekend I will post a call for testing with details on what has changed and what bugs were fixed. Stay tuned!

Now, you can easily toggle attributes:


my $object = Foo->new;
$object->flags_toggle('shiney');
$object->flags_has_flag('shiney') #true


Easy! I have some changes I want to add to this still, mostly support for renaming the provided methods (_has_flag and _toggle), and adding support for inspecting a bitmask in human form. But it’s really coming a long, and I look forward to getting this onto CPAN!

I’m pleased to announce that Google’s Summer of Code has announced which projects have been accepted. I’m pleased to let you know that our own Oliver Chalres and Lukáš Lalinský have been accepted to both work on our Next Generation Schema. Congratulations to both of you — this should be an exciting summer!

As the first act of getting ourselves organized for Summer of Code, the three of us have agreed to the following milestone schedule over the summer:

May 25: Object model and read-only user interface in place. This is essentially equivalent to Lukáš’ NGS-p implemented in Catalyst/Template Toolkit based on Oliver’s work from the last year. With this milestone users will be able to convert an existing database to the NGS schema and be able to browse the data in the new schema via the read-only user interface. No editing will be possible at this point in time.

June 29: The basic types are in place for editing artists, labels, and release-groups. Release and track level edits will not be complete.

July 27: Release and release related edit-types will be in place, but without a complete UI. The release edits will take a lot of work to get right so, we’ll have these edits in place, but may not be able to finish a working UI for them.

Aug 31: All remaining edit types are in place and the NGS enabled server enters a final beta phase.

Note that SoC doesn’t officially start until May 23 — we’re not wasting any time — in face our first milestone is due 2 days after SoC starts. Can you tell we’re serious this year?

In the past month there has been a ton of activity behind the scenes here at MusicBrainz and I can finally give a cohesive update on our plans for the next few months.

The much anticipated Release Groups release has been coded by Lukas in a weekend code sprint based on the old Mason codebase. Even though we had declared the old codebase as end-of-life, we have decided to push release groups out using the older code in order to satisfy the needs of the BBC and other customers. As part of this release, I will also add ISRC support and include a handful of bug fixes. Expect this release in May — I’ll post again when the date is firmly set.

And what is even more exciting is that we’re about to start work on our much anticipated Next Generation Schema (NGS). Discussed and planned over and over again, we’ve finally settled on an approach that appears to make everyone happy. As part of SoC, we’re likely going to accept Oliver and Lukas’ proposals to work on NGS over the summer. The goal is to implement all of the new schema in one release based on the TemplateToolkit/Catalyst work that Oliver has been working on since last summer. We’re going to take a step back and create a new object model/schema and then glue the TT UI onto the new object model.

The schedule puts the TT/Catalyst/NGS release into final beta test on August 31, with a release following in 15-30 days after that date. Please note, however, that there will be no other release based on Oliver’s TT work before NGS is release in September. We had to skip that release in order to pull in the schedule to make the target date of August 31.

I am quite excited by the work that is being done in the server area right now — we’re on our way to get past some significant hurdles. Just yesterday I got a first glimpse at the MusicBrainz site partially translated to Dutch — startling at first, but quite exciting when you think about it.

Many thanks to Matt Wood at the BBC for having the patience and dedication to work with MusicBrainz. Many thanks to Lukas for the coding sprint to get Release Groups off our collective plate. And of course many thanks to Oliver, Nikolai, Brian for your continuing hard work on TT. And thanks to everyone who has been supporting this team over the past few months.

Great news! Google has accepted us for another round of Summer of code!

Today’s the day - our wiki is being migrated to MediaWiki.  The old “moin” wiki is now read-only (and will remain so, at least for a few months), and is available on oldwiki.musicbrainz.org.  The new wiki, once all the data has been migrated across, will be at the usual address.

As soon as the migration is complete, I’ll switch wiki.musicbrainz.org over to point to MediaWiki.

Unfortunately it won’t be possible to also migrate the user accounts from moin to mediawiki, so regrettably this means that once mediawiki us up, you’ll have to re-create your accounts.  Sorry about that.

Update: the switch has been made - if you have any questions to ask or problems to report about this, please see the WikiMigration page.  Thanks!

Google has announced that Summer of Code will be happening again. We’re in the process of applying to participate again and I could use your help in coming up with some ideas for Summer of Code. Please take a look at our as of yet blank ideas page and add new ideas of your own.

Thanks!

I just rolled out a few bug fixes and enhancements to the main server:

• support for UserVoice feedback button in the menu bar
• Fixed bugs: #4023, #4156, #4858, #4588, #4703
• AR Add Release Edit search on pre-defined edit searches and advanced edit search
• Magnatune Coverart support

Many thanks to Luks, Nikki, Brianfreud and Navap for helping with these fixes/improvements.

Sander van Zoest and Dan Brickley have been prodding me to officially post about Flickr Machine tags — using Flickr machine tags you can now tag your photos on Flickr to refer to specific MusicBrainz entities. This is how to create MusicBrainz machine tags on Flickr:

• musicbrainz:artist=<MBID>
• musicbrainz:release=<MBID>
• musicbrainz:track=<MBID>
• musicbrainz:label=<MBID>

For more details see the official wiki page: FlickrMachineTag

In working on figuring out how to integrate UserVoice into our workflow its become painfully clear that we need to have some people take “ownership” over a few portions of MusicBrainz. Back in the day when Cristoph König (Don Redman) was our Wiki warden, our wiki was well groomed and worked smoothly. Alas Cristoph has been swallowed whole by a University in Germany and we may never see him again.
Its time to find volunteers who are interested in taking on personal ownership of these aspects of MusicBrainz. I’m looking for 2-4 volunteers, one for each of the following areas described below. However, please note that these people are not expected to carry out the bulk of the work that needs to be done to get these projects into an improved state. I envision each of these people being motivators and leaders who can focus and encourage the efforts of other volunteers to help them achieve their goals — there is a big difference between a leader and a workhorse.

1. Wiki: This person should be well versed in Wikis and understand their general nature. A “Wiki Warden” should be willing to help with the upcoming wiki migration to MediaWiki and have a vision for how our wiki should be organized and cleaned up. This person should be willing to do more work initiallycleaning up the wiki and then spend a few hours every week watching over activity in the wiki and gently steer the wiki into a direction of sense and overall usefulness.
2. Documentation: This person should be working with the Wiki Warden (it could even be the same person) to coordinate the creation/maintanance of Wiki pages for use with our WikiDocs documentation system. This person should be have an overall vision for how to organize our documentation and to move from our current state to a more organized and useful documentation system for MusicBrainz.
3. Trac: This person should be familiar with trac and work to remove the cruft that has accumulated in the bug tracker. We need someone to close old bugs that no longer apply, highlight bugs that are important and generally reduce the duplication present in the system. It would be best if this person could also take part of the weekly developer chats in order to stay in tune with the development process.
4. UserVoice: While a lot of people have volunteered to take part in working with UserVoice, we need one person to take the lead and be in charge of the process. I would love it if this person could help us to settle on one of the proposed UserVoice workflows.

I’d like to stress once again that I am not looking for people to do a lot of work — I’m looking for leaders who can do a little work, but motivate others a lot of help out and create a thriving sub-communities that allow MusicBrainz to become more concise and cohesive. If you’re interested in one or more of these positions, please post to the comments.

This week MusicBrainz experienced problems while trying to deliver mail to gmail.com / googlemail.com. The problems started on Tuesday morning (UK time).  On Friday morning the problem was identified as a broken DNS server, which was then fixed, thus resolving the problem.

Regrettably this means that some messages that MusicBrainz should have sent are now lost.  The number of lost messages is approximately:

• 103 ’subscriptions’ messages from Tuesday
• 61 ’subscriptions’ messages from Friday
• 297 other messages (new user signup, edit notes, etc)

Please accept our apologies for this error.

After considering all of the options and taking in tons of feedback from our developers, our community and our customers, I’ve finally settled on the following road map for the MusicBrainz server. The plan allows us to follow the release early, release often methodology and should hopefully make most people happy:

TemplateToolkit/Catalyst Release

• Date due: Late March/April
• Non schema change release
• Deployed as beta.musicbrainz.org starting in early march
• 100% features of 2008-11-23 release supported.
• Full Guess Case supported, but no new features
• Many UI improvements, including a generally speedier interface
• Internationalization support: Support will be under the hood, but we may not immediately support new languages.
• Ease of installing mb_server: Use modern tools to draw in more server developers.
• Search improvements: CD Stubs. Possibly: Advanced Relationships, edits
• Closing a whole host of old bugs, introducing many new ones.

This release doesn’t add many new features, but in general we feel that the user interface experience will improve so much that our end users will be happy with this new release. Also, since we’re keeping to a non-schema change release, we will be able to load this release and let people test with live data on beta.musicbrainz.org. I believe this well get more people to come help us test the new interface and hopefully have a smooth rollout of the new features.

NGS Release

• Date due: To be determined — hopefully sometime this summer
• Major schema change release
• Full NGS object model under the hood.
• Expose ReleaseGroups, if not other new NGS features
• Keep old edit system in place

The major piece of work in this release is the new NGS schema/object model. How we graft the existing edit system on top of the new schema remains to be seen — this task contains many unknowns, which is why it was yanked from the upcoming release. We’ll end up doing more work overall by having the TT/Catalyst release first, but this approach allows us to release early, release often.

NGS Improvements releases (may be one or more releases)

• Date due: weeks after NGS
• Non-schema change releases
• Bug fixes
• Incremental improvements in NGS
• Exposing new NGS features if the NGS release didn’t expose 100% of the new features

Depending on how we work out the NGS release, I can see smaller follow up releases that improve the NGS release or expose more portions of NGS that weren’t previously exposed. Exactly how this shapes up will not become clear until we near the NGS release.

Edit System Rewrite

• Timeframe: To be determined
• Schema change release
• Improve may aspects of our edit system.

Our edit system has been straining under its current load for some time. This release will throw out the edit system entirely and build a new more flexible system that allows the user to change more data with fewer edits and allows the user to find edits easier.

If you have questions on the roadmap, please post them in the comments.

A few people have been commenting on how MusicBrainz has a few bugs that bother them on a daily basis, yet these bugs are never fixed. From a developers perspective its really hard to see which bugs in our large bug list really matter to end users — its hard to figure out which bugs should be fixed first. Rather than fixing the bugs that affect most people first, developers tend to fix the bugs that have the most insistent users shouting for those fixes. Unfortunately fixing bugs for overly vocal users may not be the same as fixing bugs that will improve life for the most people.

Warp suggested that we check out Get Satisfaction, while others suggested using UserVoice. Both of these services provide an intermediate layer between the developers and the end users to provide feedback about the bugs that are important to them. End users can enter the bugs that bother them and everyone can vote on those bugs/issues. While we already have a bug tracking system, this system is geared to the less technical users out there — not everyone loves entering bugs into trac. Plus UserVoice’s voting mechanism allows the most important bugs to rise to the top of the list.

I’ve explored UserVoice a little since they offer a free plan for Open Source projects. (Not to mention that their site is in Orange, which is a big draw for me. ) It seems that it would be easy to integrate this with MusicBrainz. But, it seems that this is one more chore for someone to manage and the last thing I need is to take on another chore. I’m willing to do the technical integration, but I would need some help from volunteers to manage the day-to-day operations of UserVoice. I envision volunteers to pay attention to the data that collects at UserVoice and maintain a mapping between UserVoice issues and actual bug/enhancement tickets inside the MusicBrainz bug tracker. If you already play with trac and help us maintain it, this might be another aspect in which you could help MusicBrainz.

I’d like to know:

• Is using UserVoice a good idea?
• Do you see any potential problems in using it?
• Would you be interested in being an Admin on UserVoice to help manage MusicBrainz’ UserVoice site?

Please let me know in the comments!

Timothy Lee says:

After a month of trying universal builds through macports unsuccessfully, using a tool named ‘unify’ (h/t John) to lipo i386 and ppc arch bins together unsuccessfully, and bugging just about everyone I know to use their i386 tiger machine (I have an i386 leopard and a ppc tiger) unsuccessfully I’ve made the decision to halt my progress on trying to deliver a UB.

What I do have though is two builds. I have one PPC build that was built on a tiger machine and one i386 build that was built on a leopard machine. I have not had a chance to test my i386 leopard build on a tiger machine (read above failure to ‘borrow’ a tiger machine) but there is a chance it may work. If you feel like you have a better handle on the ‘lipo’ process, please, take my builds and smash them together and let me know! (It may be desirable NOT to deliver a UB as
each separate build of Picard is fairly large).

Timothy is looking for feedback on these builds. If you’ve been waiting for a complete version of Picard that includes working PUID generation, then please try these builds:

Picard for OS X Intel i386 (md5)
Picard for OS X PPC (md5)

If you have problems running these please enter a bug report and use the component “Picard Tagger (Mac OS X Packaging)“. Thanks very much Tim and everyone else who has helped along this somewhat frustrating process.

Here is the MetaBrainz Foundation annual report for 2008:

Introduction

2008 was a busy year for MusicBrainz: Our traffic doubled from January to December, we grew from 8 to 13 servers and we earned $35,000 more than in 2007. Most importantly we started 2008 with only one full time employee and ended the year with 2 and a half full time employees. Between having more employees and having three students work on Google Summer of Code projects, we’ve had more code written in 2008 than we have ever had before. A lot of these projects were long-term projects that would take us well into 2009 before seeing benefit from this work. Consequently we had only one server update in all of 2008, but we’re expecting to have about 3 major releases this year.

Despite widespread upheaval in the world economy, MusicBrainz managed to grow and stay consistent all through 2008. We added Amazon.com, MetaWeb and Cloudspeakers as our major customers and Google sponsored MusicBrainz with another $30,000 donation. Finishing 2008 with $71,000 in the bank MusicBrainz starts 2009 strong and healthy.

Profit & Loss

In 2008 the foundation took in $126,442.26 and spent $94,717.79 for a total profit of $31,724.47. (non-profits can still earn a profit, but that profit must be reinvested into the company, not paid out to its officers and directors.) A detailed break down of where the income came from and where it went to is shown below:

Income

Direct donations	$30,219.00
PayPal donations	$13,271.25
Consulting (GSoC Mentor income)	$1,500
Live Data Feed licenses	$60,472.56
CC Data Licenses	$2,400.00
Interest	$1,211.92
Amazon Associates	$1060.72
CD Baby affiliate	$11.00
Tagger Affiliates	$16,294.94
Bank Credits	$0.87
Total income:	$126,442.26

Expense

Bank fees	$240.90
PayPal fees	$1,546.61
Rent	$2,856.00
Hardware	$9,107.98
Travel	$2,456.61
Internet	$275.41
Marketing	$1,564.69
Development	$60,415.16
Supplies	$38.23
Gifts	$100.00
Events	$792.17
Hosting	$11,904.50
Filing Fees	$45.00
Software	$239.92
Entertainment	$75.15
Insurance	$2008.00
Accounting	$965.00
Shipping	$86.46
Total expense:	$94,717.79

This Profit & Loss shows:

• It cost $21,012.48 to host MusicBrainz in 2008 — this includes hardware and hosting costs.
• PayPal donations, which represent end-user support for MusicBrainz, came to $13,271.25, about 10.5% of our total income. It has been our goal to have the end-users pay for hosting MusicBrainz, but this year we fell significantly short of that goal due to greatly increased hardware costs. However, if we take into account the income from the Tagger Affiliate program, then the picture improves quite a bit. Tagger Affiliate income is MusicBrainz’ 10% revenue share of end-user registrations of third party applications that use MusicBrainz. Counting these, our end user contributions total $29,566.19, which exceeds the cost to host MusicBrainz by over $9,000.
• As expected, salaries continue to be the largest expense for The Foundation. In 2008 it paid $58,000 in salaries to Robert Kaye, which was still below a reasonable salary level for an Engineer/Executive Director in California. Its first non-founder employee Oliver Charles was paid $2,415.16 for just over three months of work.
• $62,872.56 was earned licensing the MusicBrainz data for commercial use, which represents 49.7% of the total income. This is up over 9% from last year.

In 2008 the foundation spent, $21,012.48 on hosting and hardware costs and served out 1.3 billion web hits and 790 million web service hits. Calculating a cost per hit, we find that we spent $16.21 per one million web hits and $26.58 per one million web service hits. These values are down significantly from the 2007 values of $17.89 and $33.58 respectively.

Balance Sheet

The balance sheet for the end of 2008 showed the MetaBrainz Foundation with $39,525.20 retained earnings, a net income of $31,249.67 and total cash assets of $71,249,67.

Traffic

The following chart shows our traffic growth for 2007 and 2008:

The blue line represents the overall number of hits to musicbrainz.org. The red line shows how many of the overall hits were web service (API) hits; as you can see towards the end of 2007 half of our overall hits were web service hits. Towards the end of 2008 our web service hits accounted for 60.9% of our overall hits, up from 50% in 2007.

Top data contributors

Top editors:

drsaunde	141064
ojnkpjg	40796
murdos	33517
voiceinsideyou	32660
knakker	31598
brianfreud	30242
teleguise	29756
bgibbard	23676
MrH	22521
Isomer	21203
cooperaa	19808
mudcrow	19162
fred576	17634
chiark	17561
StoneyBoh	16637
zos18	16466
Jormangeud	16264
leivhe	15983
Mistoffeles	15710
Creap	15373
jesus2099	12990
Dr. Default	12099
montesquieu	11810
infofarmer	10990
nikki	10015

Top voters:

ojnkpjg	35834
drsaunde	35374
voiceinsideyou	28376
chiark	20704
mudcrow	19892
murdos	14793
leivhe	12864
bogdanb	12560
bgibbard	11898
cooperaa	10954
symphonick	10642
teleguise	9778
vincentrichter	8668
MClemo	8302
DrMuller	6847
bawjaws	6208
infofarmer	5861
PhantomOTO	5747
Kerensky97	5740
debris77	5611
lukz	5460
dmppanda	5163
krazykiwi	5097
dj_empirical	5035
Atedos	4970

A big thank you to all of the editors/voters who contributed! MusicBrainz would be nothing without your hard work!

Server farm

MusicBrainz has grown to 13 machines in service:

From the top, going down:

• moose: Our database server
• scooby: Our aging catch all server: blog, wiki, forums, mailing lists, etc
• catbus: Raw datavbase server (raw tags, collections, etc)
• bender: Former TRM server, now idle cold spare machine
• blik: Backups and memcached
• stimpy, dexter: Web, web service servers
• cartman: Search server
• wiley: New catch all server: SVN, trac
• lenny/carl: Redundant network gateways
• misty/jem: Search servers

MusicBrainz currently requires somewhere between 3 and 4 mbits of bandwidth per second and draws 18 Amps of current for a power consumption of nearly 2,000 Watts. MusicBrainz physically occupies 20Us of space (half of a rack) at Digital West in San Luis Obispo, CA.

Words of Appreciation

2008 was an exciting year for MusicBrainz and it could not have happened without the countless volunteers who have spent inordinate amounts of time and effort working on MusicBrainz. It is not possible to thank everyone who has contributed — for starters see the list of top editors and top voters. There were so many people behind the scenes or at partner companies that are not readily apparent from the outside.

MetaBrainz would like to thank Matthew Wood and his entire team at the BBC; through Matthew’s efforts the relationship between the BBC and MusicBrainz has been flourishing. Wendell Hicken and Sergey Borisov at MusicIP are the stars at MuiscIP who look out for MusicBrainz every day. Lukáš Lalinský, Oliver Charles, Aurélien Mino and Brian Schweitzer — your contributions to the MusicBrainz server cannot be underestimated! Dave Evans, you’re MusicBrainz’ unsung hero for all of your efforts behind the scene — MusicBrainz keeps running because of you!

Thanks to everyone who helped MusicBrainz in 2008 — MusicBrainz could not exist without you!

Finally, in 2008 quite a few contributors donated chocolate to MusicBrainz as a form of appreciation. Last year working on MusicBrainz was helped along with chocolate from Germany, Switzerland, Finland, Norway, the United Kingdom. Thanks to everyone who sent in chocolate!

(Yes, this report is a year late — I had spent a lot of time on the annual report for 2006 and got so little feedback on it that I assumed no one cared. But now in 2008 I’ve been getting many more requests, so I’ve gone back and with the help of Pavan compiled the stats for 2007.)

Profit & Loss

In 2007 the foundation took in $89,645.59 and spent $64,543.84 for a total profit of $25,101.75. (non-profits can still earn a profit, but that profit must be reinvested into the company, not paid out to its officers and directors.) A detailed break down of where the income came from and where it went to is shown below:

Income

Direct donations	$30,205.00
PayPal donations	$14,575.59
Consulting (GSoC Mentor income)	$1,500
Live Data Feed licenses	$30,340.47
CC Data Licenses	$1,900.00
Interest	$357.45
Amazon Associates	$714.39
Tagger Affiliates	$10,052,69
Total income:	89,645.59

Expense

Bank fees	$64.65
PayPal fees	$1,570.92
Rent	$2,779.00
Hardware	$732.06
Travel	$2,933.49
Internet	$89.01
Marketing	$765.00
Development	$39,500
Supplies	$56.88
Events	$400.92
Hosting	$12,240.00
Filing Fees	$25.00
Software	$-41.45
Entertainment	$27.32
Insurance	$1,996.00
Accounting	$1,200.00
Total expense:	$64,543.84

This Profit & Loss shows:

• It cost $12,972.06 to host MusicBrainz in 2007 — this includes hardware and hosting costs.
• PayPal donations, which represent end-user support for MusicBrainz, came to $14,575.59, 16.3% of our total income. The Foundation aims to have end-user donations pay for the actual cost of running the service and the end-users did in fact pay for slightly more than what it cost to host MusicBrainz in 2007!
• As it is with a lot of companies, salaries accounted for the largest slice of expenses. The Foundation paid $39,500 in salaries to Robert Kaye in 2007, which was still far below a reasonable salary level for an Engineer/Executive Director in California.
• $32,240.47 was earned licensing the MusicBrainz data for commercial use, which represents 40% of the total income.

In 2007 the foundation spent $12,972.06 on hosting and hardware costs and served out 725 million web hits and 386 million web service hits. Calculating a cost per hit, we find that we spent $17.89 per one million web hits and $33.58 per one million web service hits. These values are up slightly from the 2006 values of $17.56 and $30.58 respectively.

Balance Sheet

The balance sheet for the end of 2007 showed the MetaBrainz Foundation with $14,323.45 retained earnings, a net income of $25,101.75 and total cash assets of $39,525.20.

Traffic

The following chart shows our traffic growth for 2007 and 2008:

The blue line represents the overall number of hits to musicbrainz.org. The red line shows how many of the overall hits were web service (API) hits; as you can see towards the end of 2007 half of our overall hits were web service hits.

Top data contributors

Top editors:

drsaunde	125665
ojnkpjg	106211
brianfreud	102090
mudcrow	44423
murdos	32111
teleguise	26541
dmppanda	23264
cooperaa	16705
Schika	16420
zout	16160
voiceinsideyou	15268
artysmokes	13380
Kerensky97	12174
foolip	11403
lukz	11031
bench12345	10014
AnAlach	9323
gioele	8742
headlocker	8450
Pianissimo84	8364
SenRepus	8183
leivhe	7815
Liff	7748
lytron	7740
zos18	7533

Top voters:

drsaunde	47850
ojnkpjg	41365
brianfreud	27382
voiceinsideyou	19874
cooperaa	19348
headlocker	16183
lukz	13792
mudcrow	12861
gioele	11259
cadalach	10677
bogdanb	10496
CatCat	9214
symphonick	8767
teleguise	8571
Dr. Default	8313
BrianG	7936
crazee_canuck	7842
keschte	7838
cybercox	7802
Xaiver	7699
Pianissimo84	7175
murdos	7165
artysmokes	6495
helver	6349
debris77	6276

A big thank you to all of the editors/voters who contributed!

After the last release our new spiffy lastupdate feature that was supposed to keep track of when data in the database changed brought our database server to its knees.
The problem was that if we had an artist, we would also go update all of its releases. And if you updated a release, the artist and label had to be updated. We turned off all of the cascading updates and our server went back to normal.

Thus, for the time being updates will not cascade. If you update an artist, none of the releases or associated labels will update. In the future we will see if we can improve on this.

The good news in this is that the Dashboard feature is still useful, so I’ve turned it back on and made it live on the main server.

P.S. We just passed our 10 millionth edit!!

A belated Happy New Year to all MusicBrainzers out there!

If things seem quiet in MusicBrainz land, don’t be fooled! The style mailing list is quite busy with activity as the style process has been rebooted. Development is happening at a frenzied pace as most developers are focusing on the server rewrite that has been in progress since summer last year. In a sense its already fulfilling one of its goals: More developers are taking a look at the improved, easier to understand and easier to install codebase. I don’t think we’ve ever had this many developers tinkering with the server source code!

We’re now in the process of coordinating a growing team of developers and charting the course for future releases. Its still too early to really tell what the next release will include exactly, but once a concrete plan starts shaping up, I’ll post an update here. Also, another team of people is hard at work at creating a new binary install of Picard for OS X. You can follow all the happenings on the developers mailing list.

Finally, the MetaBrainz Foundation, the legal entity that pays the bills for MusicBrainz made it through another year of being self sufficient! In 2008 the non-profit earned $126,442.26 and spent $94,717.79. Now we have $30,000 more in the bank than we did this time last year. This finally gives us money to spend on development to move MusicBrainz along! For all the details on how we earned/spent our money, check out our finances page and our 2008 Profit and Loss statement.

Finally, the BBC continues to tout the virtues of MusicBrainz: BBC Music talking semantics

Onward into an exciting 2009 we go!

A few of you have reported problems with connectivity to the MusicBrainz site in the last week. With help from various people in Canada and Europe we’ve been able to provide information for Digital West (our provider) to fix the problem. As of 11:38 PST the problem was fixed.

If you are still experiencing problems, please post a comment here or open a bug in our bug tracker.

Thanks and sorry for the inconvenience!

I’m pleased to announce that I signed a new customer in the Netherlands yesterday! I met the Cloudspeakers team in Utrecht and spent a day discussing their business and how Cloudspeakers can work with MusicBrainz in the future.

After many conversations during the day, we ceremoniously signed the data license contract over an elegant and extremely tasty Indonesian meal. A day well spent!

A big thanks to Chris, Stan, Adriaan, Charlie and Francis! Thanks for inviting me, paying for meals and even my flight to the Netherlands. I look forward to working with you!

I come bringing good news! With the latest version of the server now out in the wild, we’re ready to move on to the next stage of MusicBrainz development. But first a quick refresher…

You may recall, many moons ago I (Oliver Charles aka aCiD2) began work on moving the mb_server codebase from our own in-house framework, to the tried and tested Catalyst framework - along with separating out the HTML into separate Template Toolkit templates. Well, after what seems like an age, it’s finally got to the time where I can start getting some critical feedback from the most important people - you!

As from now, test.musicbrainz.org is now running the development branch of this work. It’s important to realise that this new codebase currently has no javascript support. This decision was made because it’s very important we get the website fully functional, and then add bells and whistles on later. We’re starting from a mostly clean slate, so there’s a lot of chance of things breaking, and JavaScript was likely to be just one more headache.

Oh, and I’ve never deployed a server like this before, so please bare with me while I work out any problems running the server. I’m going to London tomorrow and coming back Saturday evening (slightly bad timing, I’m aware) - but I’ll do my best to check any messages that come my way!

However, before you jump straight in and overload us with work - I’d like to lay down some guidelines for providing us with feedback. This will (hopefully) ensure that we can see to these bugs as fast as possible.

Where to report:

Standard practice- report at our bugtracker - bugs.musicbrainz.org

What should you report?

The most important things to report are actions that cause errors to occur, invalid behaviour or features that are simply not available, but are from the main server. You should also report typos and other visual problems - but I will be encouraging people to help fix these themselves (more on this later!).

What information should you provide?

The most critical information is that you can provide us with as much context as possible. Please let us know:

• Any steps to reproduce the problem
• Whether you are logged in or not
• The address of the page that caused the problem
• As much information as possible from the top of the error page

The last point relates somewhat to Catalyst. Catalyst features improved error handling and can provide us with a stack trace. You should try and include this stack trace, and the errors at the top. Chances are, we’ll be able to reproduce this from the information you provide - but if not, the stack trace gives us one more pointer to where the problem is

How should you organise the report?

Generally speaking, just try to fill in as many of the fields as possible. I’ll be reading every single report that comes in, and re-filing it myself where necessary. Ideally, set the component to ‘MusicBrainz’, the milestone to ‘Server: TemplateToolkit’ and assign the bug to me

So…

Let the bug crunching begin!

We just completed pushing the latest changes out to the main servers! We had a bit of a bumpy ride to roll out the upgrade — we’re noticing quite a few problems with collections right now and the Last Update feature brought our database server to its knees. As a result, we’ve disabled the Dashboard — we’ll re-enable it once we figure out what the problem is.

If you encounter a problem with the server, please file a bug report and select the 2008-11-23 version. Also, please check the open bug list to see if your problem has been reported before.

For a complete list of things that changed for this release, please see the release page on the wiki.

This massive release was brought to you by the tireless efforts of: Luks, Murdos, Djce, Jugdish, Acid2, Niklas and myself. Loads and loads of good testing came from Voiceinsideyou and Nikki. Thanks to everyone who helped with this release!

Also, if you’ve used Jugdish’s enhanced voting GreaseMonkey script, please disable it as it may cause problems since that functionality was included in this server release.

Due to a scheduling conflict we were not able to get the release completed today. We ran out of time today to get it done. We’re going to attempt again tomorrow at Noon PST, 2000 UTC, 2100 MET.

Sorry for not getting this done today.

I’d like to remind everyone that the next server release is scheduled for this Sunday November 23. We’re still working on scheduling for the release, but it will happen on Sunday. Expect the MusicBrainz to be unavailable for at least 1 hour.

Also, please be aware that with this release the TRM server will be decommissioned — the Classic Tagger will then be officially deprecated, but it should continue to work — even without TRM support.

I lately discovered CuriousInventor as they sell some quite interesting diy kits like the touch strip LED display that looks perfect for synthesizer controlling to me (Hope to get one for christmas). But the really interesting thing to me is that you get a PanaVise Junior Clamp for free if you link them from a page with Google PageRank of 2 or higher (this page had 3, when I checked). So lucky me, I get a gift worth 18$ at their shop for free. Who said making profit from the web would be hard? ;)

Thanks to the tireless efforts of Matthew Wood, the BBC has officially hired Jason Emmett to work full time on the MusicBrainz server!

Jason, who will be working in the BBC offices in London will be working with Oliver to finish the port to Template Toolkit branch. After that both of them will tackle the Release Groups that we’ve deemed to be a worthy intermediary step. A few weeks ago the MusicBrainz server had only table scraps of love, moving it along quite slowly. Today we have nearly 1.5 full time people working on the server source code. This gives me tons of hope that we can shoot for doing 3-4 releases in 2009.

Thank you so much for all of your efforts, Matthew Wood. Thank you for all of your support BBC! Welcome on board Jason!

Murdos, Niklas, Luks, Dave Evans and I have all been working on the upcoming server release and we would like to invite you to help us test this massive release. Most of the features for this release have been debugged, but a few bugs are still outstanding.

The test server has been updated with all of the latest features and is ready for you to log in with your normal user account with the password ‘mb’. The list of features in this release is described in this blog post and this post.

Submit new bug reports to our bug tracker. Also see, the milestone for this release to see which bugs have been fixed and which bugs are still open. Please check the open bug list before reporting a new bug.

Thanks to everyone who helped take part in the development for this release!