Sunday, November 13, 2011

Composing with Linux -- Part 2: Developing the Idea


A few people have asked me about my workflow for composing so I thought I would collect some of my technical notes and share them.  Having spent 2 years taking classes with Berklee Music Online, I was able to get a workflow fairly streamlined for my purposes. I am breaking this up into multiple articles else it would be way too long....

This is part 2 -- Developing an Idea.


Ok, I've gotten an idea for a piece of music... what do I do to bring it to life? A lot depends on what kind of music I am writing -- if I am doing an electronic ambient piece or a crunching metal tune, I typically start with the sequencer (Rosegarden) and start working out the rhythm section parts and will work out notation (if needed) after the fact. A film score, however, while usually a symphonic piece and ultimately following the ideas outlined below, requires some additional pre-production work which is a separate topic altogether.

If I am working on a symphonic piece, however, I will typically start with the notation first. For this, I use an application called Lilypond. It's renowned for producing fantastic looking scores, but unlike other applications like Sibelius or Finale, Lilypond is not a graphical application. Instead, you produce a source file using a text editor and then generate the score from that. This lets you focus on the content of your music without getting bogged down in the layout (much the way that HTML works). I actually kind of compromise here and use a graphical front end for Lilypond called Frescobaldi. Frescobaldi is kind of a hybrid way of producing the source code as well as having the generated score in front of you. It's not fully interactive like Sibelius, but more along the lines of an integrated development environment (IDE) used by programmers for software projects.



Since Lilypond is text based, it allows you to "include" files in other files, just like with programming languages. This is a very nice way to organize your music, so you can have your individual instrument's notation in separate files, and then create a master document that pulls the data in from the other files. There are many things you can do with Lilypond, which I won't go into here, but it is a very flexible scoring system once you learn how to use it. Lilypond is available for all platforms -- Linux, Windows and OS X. Frescobaldi is only available for Linux but a new version is due out in December 2011 that will run on all platforms. There are other frontends for Lilypond, too, see the Lilypond website for more details.

For larger orchestral scores, I will usually start with a "sketch", which is a simplified score with staves for woodwinds, brass and percussion, split into treble and bass parts, as shown in this snippet:


Here I can work out ideas and harmonic arrangements quickly, indicate ideas for percussion, make notes to myself, and so on, and develop the overall structure of a piece, without getting too wrapped up in the orchestration, although some basic orchestration will be done here. From this basic sketch, I can move on to creating a fully developed orchestration of the music. I won't get into orchestration techniques here, as it is a pretty deep and complex musical topic -- maybe a topic for a separate thread of posts!

Lilypond will also generate a MIDI version of the score, which you can play back with a sequencer (Frescobaldi has an embedded MIDI player, you can see it in the lower left corner in the screenshot above). This MIDI playback is used only as a reference, so I can hear how the harmonies work and if the melody is what I want it to be. Lilypond is not good for producing performance data, nor is it intended to be. One drawback with Lilypond -- a fairly major one, in my opinion, which I hope is addressed at some point in the future -- is that it can only use a single MIDI output device (16 channels total), so the way I work around this is to create each section of the orchestra as multiple voices of a single channel, so all of the brass instruments will use the "brass section" General MIDI (GM) patch, all of the strings will use the "string section" GM patch, and so on.

One the MIDI is generated, I can pull it into the sequencer for getting a good performance, which is the subject for the next post.

Composing with Linux -- Part 1: Software and Hardware

A few people have asked me about my workflow for composing so I thought I would collect some of my technical notes and share them.  Having spent 2 years taking classes with Berklee Music Online, I was able to get a workflow fairly streamlined for my purposes. I am breaking this up into multiple articles else it would be way too long....

This is part 1 -- Software and Hardware.

Open Source

I use Linux for all of my audio work (with some exceptions noted below), which also means I primarily use open source software to produce my music. A lot of this software may not have all of the bells and whistles provided by commercial software apps, but it suits me very well. And I daresay I may be one of the first people, if not the first, to get certified through Berklee for film scoring and orchestration using open source software.

There are still pervading myths that there are no applications for Linux, Linux isn't good for multimedia, there isn't any hardware support for Linux, etc. These myths may have been true in 1997 but they aren't true today. I have been using Linux as a hobbyist since 1996 and professionally as a software engineer since 1998, and it has evolved quite a bit since those early days. In fact, not only do I use Linux for music production, I also use it for digital art and animation (I will save that for another post). There's no question that using open source software is also more affordable for those on a tight budget (less money spent on software means more can be spent new guitars on high-end hardware).

I use Ubuntu Studio Linux, v. 11.04. In the past I've also used Gentoo and Fedora + CCRMA, they all worked very well. However, Ubuntu Studio is one of the most streamlined Linux distros and everything I've plugged into a machine running Ubuntu Studio just works, from scanners to wacom tablets to various USB MIDI devices.

The primary open source applications that I use:
One piece of software that is important for my setup is JaCK (http://jackaudio.org/), which should be the heart of any professional audio workflow with Linux. It's an audio routing daemon which allows applications to send synched audio to each other, it's a fantastic way to have multiple apps cooperate in recording and mixing, even apps that are on other machines on a local network.

I also use a 64-bit Windows 7 machine as a sample host -- this is the exception to the Open Source rules. The software used on Windows:

  • Kontakt 4 (with lots of 3rd party libraries)
  • EastWest/Quantum Leap PLAY (Symphonic Orchestra, Silk, many others)
  • IK Multimedia SampleTank (and SampleTank-based apps like Philharmonik)
  • FXPansion BFD2 Drum Sampler
  • Reaper 64-bit (used only as VSTi Host)
  • Various other libraries and tools
Hardware

There's no question that you need good hardware if you want to do professional or semi-professional audio. Consumer-level sound cards typically have cheap DSP chips and are not suited for the low-latency needed for professional multitrack recording. This especially includes the onboard audio interfaces that come on most commodity computers. Many, if not most, USB audio devices are also not well suited for multitrack recording either, especially when realtime monitoring is needed. Dedicated PCI or Firewire interfaces are still the best choices for professional audio, and this applies to Linux as well.

The most popular sound architecture on Linux is ALSA (http://www.alsa-project.org). It supports many different kinds of hardware, providing there is a driver available for the hardware. Most manufacturers do not develop Linux drivers, but many have provided open source developers with specifications for their devices upon request.

My Ubuntu Linux machine uses an RME HDSP Multiface II audio interface. This is an external unit with 8 analog inputs and outputs, plus one stereo S/PDIF in/out and one LightPipe in/out (4 stereo pairs or 8 mono). It connects to the computer using a dedicated PCI card. RME devices are among the best supported on Linux and this device has worked wonderfully for me. It provides zero latency hardware monitoring, which is essential for any kind of multitrack recording.

For monitoring, I use M-Audio Studiophile monitors (BX5 stereo pairs and SBX sub-woofer). I also like to use headphones for initial mixing sessions so I can focus on the sound without distractions.

The Windows 7 machine has a single E-Mu 1010M PCI interface (S/PDIF and ADAT I/O). All of the audio from this machine is routed to the RME device and monitored via the Linux hardware.

The Linux machine, I should add, is an older 32-bit Intel Core 2 machine with only 2G of RAM. The Windows machine is a 64-bit box with 8G of RAM, with an AMD Phenom quad core processor. I definitely need the beefier hardware on Windows, since it's driving the samplers.

Next Up is Part 2: Developing an Idea

Friday, October 14, 2011

Ubuntu Studio Upgrade "Oneiric Ocelet"

(NB: As of Nov 10, 2011, I've reverted my stystem back to Ubuntu Natty (11.04), Oneiric just wasn't working well for me, TVPaint especially was troublesome under 11.10).

So... new release of Ubuntu (and related Linux distros like Ubuntu Studio and Kubuntu) came out October 13 (2011). Since I mainly use Ubuntu Studio, I'll talk about the upgrade I did on my graphics & animation workstation.

The Specs


The machine in question is dedicated for multimedia production, specifically, animation and digital art. It's by no means a top of the line machine, but an off-the-shelf HP Pavilion desktop machine I bought at Microcenter because a previous machine had died. I say off-the-shelf because I just ran out to Microcenter one evening, grabbed a box off the shelf and paid for it (avoiding a sales drone completely).

Here's the machine:

HP Pavilion a6500f desktop
4G RAM
Intel(R) Pentium(R) Dual  CPU  E2220  @ 2.40GHz
NVidia GeForce 7100 graphics w/512MB video RAM
300G system drive
1000 GB projects drive (mounted under /home)

Attached peripherals:

  • Wacom 9x12 drawing tablet
  • HP Scanjet 4370
  • Logitech HD webcam
  • External SATA HD 2-bay dock
  • Logitech USB stereo headset w/microphone
Since I don't do audio work on this machine, I use the USB headphones for sound output.

The Software

As I said before, I use this machine for animation and digital art. So the main apps installed are:
  • Blender v2.59: CG modeling and animation (native app)
  • Anime Studio Pro v6: vector animation (native app)
  • Synfig Studio v0.63.00: vector animation (native app)
  • TVPaint Animation Pro v9.5.20: traditional handrawn animation (native app)
  • GIMP: image authoring (native app)
  • MyPaint v0.9.1: natural media painting (native app)
  • Artrage Studio Pro v3.5: natural media painting (Windows app run via Wine)
  • Sculptris Alpha 6: digital sculpting & modeling (Windows app run via Wine)
  • ZBrush v4R2: digital sculpting & modeling (Windows app run via Wine)

All of these ran fine under Ubuntu 11.04, with some peculiarities with some of the apps that run under Wine, which are known issues.

The Upgrade

I kicked off the upgrade early in the evening, knowing it was going to take a while. I think it finished after about 90 minutes. No problems here, went very smoothly (which has always been my experience with Ubuntu).

The Aftermath

Previously, I'd used the default Gnome desktop. The new Ubuntu wants to use Unity instead, which I had looked at before but didn't like very much. I gave it another spin after the upgrade and am still not impressed with it. It's probably good for people first starting out with Linux, but I didn't much care of it's look and layout.

The Gnome desktop seemed "weird"... I think it's the new Gnome 3 but I didn't like what it had become,  the fonts were too big and something just didn't seem right.

I noticed XFCE was also available by default. I'd used this window manager/desktop environment in the past, it was lightweight and clean, so I ended up using that as my default desktop.

One thing I noticed immediately was that some things seemed to take forever to start up. In fact, my initial reaction was that some of the apps I tried starting, like Anime Studio Pro or Blender, might have had library issues because I clicked on their launchers and nothing happened after even 30 seconds. After another minute or so, suddenly all of the apps showed up on my screen (sometimes multiple copies as I had clicked on the launchers multiple times). Clearly something was amiss.

I did a full system reboot to get everything back to a sane state. I tried going online and found something was messed up with the network, but I quickly fixed that. All of the apps started up quickly now, with the exception of TVPaint Animation. It was still taking forever to start. I did an strace of the running process and provided the output to TVPaint Animation support. 

Eric Scholl, one of the TVPA developers, said there was an issue with the webcam and to try starting the app with the webcam unattached. Sure enough, TVPA started up instantaneously... in fact, I daresay app startup times seem faster than they did previously, now that the issues have been ironed out.

Everything seems to be working OK now -- audio works, all of my peripherals work, the Wacom tablet works. All in all a pretty smooth upgrade.

The one exception is ArtRage Studio Pro. It seems to lockup completely immediately after startup. The other apps that I use under Wine (ZBrush & Sculptris) run without issue. I'll have to investigate the ArtRage issues this weekend.

I have a second studio machine  (an old 32-bit Dell XPS model) also running Ubuntu Studio, this machine is dedicated to audio production. I'm not going to upgrade it since I am planning on replacing that machine entirely with something newer and 64-bit.

Wednesday, October 12, 2011

Where is the original content?

I am not alone in noticing the severe lack of original content coming out of Hollywood these days... remakes, reboots, re-imaginings and endless sequels. So there is no need to rehash that here.

What I have been noticing more and more, however, is a similar trend on social media sites like Facebook and Google+. A lot of people posting seem to be endlessly reposting content from other sources rather than posting something original. And by this, I mean reposting (or sharing) posts made by other people, or videos or cutesy pictures and random quotations, most of the time without any additional content beyond "This is cool" or "Awesome!" This is akin to when everybody discovered email and forwarded every single joke or cat picture that came along (this still happens, too -- it keeps snopes.com in business!)

I know I have been guilty of this myself, when I am bored or can't find something better to post about. But in the past few weeks, I have realized, if I have nothing to post about, maybe I just shouldn't post... or... better yet, write about something and post it via this blog :-)

Friday, September 30, 2011

Animation and storyboarding

Now that I am done with my music studies, I can start spending some time on my other creative passion -- animation! I am quite the fanatic of hand-made animation -- whether it's hand-drawn or stop-motion, I find it wonderful to watch and to create. I haven't produced anything in years but the time has come to do it!

As some initial inspiration, I had great joy composing music recently for a short stop-motion film by John Hankins, titled "Helping Hands":



So my first animation projects will a series of shorts under the title "Tales from the Perilous Quill" (hence the title of this blog). Tonight, I started storyboarding out the first short, which is called "The Birthday Party". I won't say much else about it except it's inspired by the cartoons of Gahan Wilson and Edward Gorey.

Thursday, September 1, 2011

Camera trickery is nothing new

109 years ago today (Sept 1, 1902) , the father of visual effects, Georges Méliès, released this cute little film:



Méliès discovered many in-camera tricks like using multiple exposures, time lapse photography, dissolves, and substitution (having an actor stand in place, the camera is turned off, and object placed in the actors hands and the cameras turned on, giving the appearance of the object magically appearing on screen). Camera tricks have been around longer than most of us have been alive, yet even now we are still often fooled by them!

When Hurricane Irene moved through the Caribbean and East Coast of the United States the week of August 21 - 27, 2011, this picture started spreading all around the Internet:


This was a scene of a flooded street in Puerto Rico with a shark swimming through it. The picture was accompanied by the warning "...this is exactly why authorities in NYC are warning people not to go swimming in flood waters after a hurricane." While I had my suspicions about this picture, it certainly looked real. It was later revealed (via snopes.com) that this was a composite photo of a great white shark from another picture and the flooded street.

We all have the tendency to believe everything we see on the Internet as being real and true. But like Georges Méliès discovered, it's amazingly easy to fool people with a camera. These days we have applications like PhotoShop or the Gimp that can turn anyone into a bedroom  Méliès.


Friday, August 19, 2011

"So that's how it was done..."

This picture was posted recently on a photoblog that usually posts pictures of strange and unusual things:


And the caption was "So that's how it was done"...

20 or more years ago, this kind of picture might not have seemed so strange to youngsters interested in visual effects for film. This picture is actually from 1979, during the post-production phase of "The Empire Strikes Back", showing the filming of the famous prologue scroll.

Some people in this day and age seem amazed that at one time, visual effects were done with real things, on a stage, with cameras, rather done entirely in the computer. I use a computer on a daily basis for media production, and I always am in awe at how they accomplished such magical imagery using rear and front projection, matte paintings, traveling mattes and optical printers for compositing multiple elements into a scene.

At the time this picture was taken, at Industrial Light and Magic, they were the best visual effects shop around. The effects might seem dull and slow compared to the dizzying computer-generated effects in current productions, but it seems there was more of an effort to make good visual effects prior to the use of modern CG effects. Many films coming out to today seem so over-produced than I can barely stand to watch more than a few minutes at a time. It's so easy now to composite CG elements into a scene to the point of visual saturation that it's virtually effortless to produce these films.

Of course, I won't completely disparage CG effects and animation, a lot of it is very good (like in "Lord of the Rings" and the Pixar films), and there's no question that some of the independent short films are just spectacular. In this day and age, even when doing traditional hand-drawn animation and stop-motion, the added flexibility and power of personal computers has made these traditional forms of animation much more accessible to the independent filmmaker.

I admit I am jealous of youngsters getting into filmmaking now... when I was 13 in the late 70s, I first started experimenting with Super 8 cameras, stop-motion, rotoscoping, and visual effects. I had to do everything in-camera because the idea of getting an optical printer to make super-8 films was beyond any possibility (they existed but were several thousand dollars). The same goes for producing audio for films, we were very limited in what we could do for music, sound effects, etc. Now, you can do everything "in the box" -- compositing, audio mixing, even creation of full symphonic scores. It's very cool.