Holy shit, this is amazing. For those that haven't written music before, previous programs are some of the clunkiest, slowest, most infuriating pieces of crap that I've ever had to deal with. This makes the process literally ten times easier, and so much more fun to use - and the MP3 export? Amazing.
I'm actually considering using Windows because of this and the new hardware. If you had told me that two days ago I and my beautiful Arch setup would have laughed in your face. Kudos to Microsoft/Staffpad, this is really great.
It is a very cool piece of tech to be sure, but as a composer who has used notation programs almost daily for over a decade, this really doesn't solve any new problems, and it isn't anywhere near powerful enough for professionals (yet), so most will be importing files from StaffPad into Finale/Sibelius/etc.
With a MIDI keyboard and some keyboard shortcuts I can enter music extremely quickly. To write a string of 8 16th notes, for example, I can press 1 button to select the note length, then press the 8 notes on the keyboard to write the notes (this is with any modern notation program). With this app, I have to draw each individual notehead, beam across the top, and then an additional beam for the 16th notes, then I have to hope I got everything in the right place so that it correctly recognizes my intentions. On that last point, anything beyond very simple scores would probably end up being a nightmare in terms of productivity. A video from last April shows a person making a very common action - beaming more than 2 eighth notes together in a 4/4 measure, but the program thinks he wants triplets, so he has to go back and erase the marking. I can only imagine how many of these "hitches" would pop up in a more complex score.
The price is fairly low, and they seem to promising more features and refinement, so it will be interesting to see where it goes. But the input method, while neat to look at, is fundamentally a regression for professionals. A Vim-style system that allows for extremely rapid entry of music would be an interesting and useful approach, but something that might not have a wide appeal. This app will be useful for those that want to jot down ideas while not at their desktop, and many people will buy it for the novelty, but I would imagine most professionals will wait.
I guess this is the WYSIWYG thing all over again- in the beginning its nice to be able to see continuously what the finished document is going to look like, but after you become very fluent with the software having to draw everything or to manually fuss with the positions of stuff becomes very tedious. I'm far from being a professional, but with MuseScore I often find that I'm barely ever looking at the app since I'm concentrating on listening to the thing I'm transcribing and just using muscle memory to adjust the note durations with the keyboard; basically touch typing I guess. The professional arrangers I know are blindingly fast with Finale or Sibelius, kind of like watching a pro draftsman using CAD software, or a real recording engineer editing in Pro Tools.
One of the reasons that I often wind up using Lilypond instead of MuseScore is when I'm writing examples like a sheet of scales. I can write a little script that emits the Lilypond notation, and then render it with Lilypond.
A couple thoughts (as someone who has spent a significant chunk of time using Sibelius):
1. The keypad / MIDI keyboard input method is definitely efficient once you're good at it, but I would say it's much harder to get used to than just drawing the notation. Drawing a dotted quarter note is natural for anyone who has taken music theory. Pressing 4 (or whatever) for quarter note, then '.' for dot, then pressing a key on a different keyboard for the pitch -- it's just a pretty intricate set of steps to go through for something that would just be a few intuitive strokes of a pen.
2. A MIDI keyboard takes up noticeable space. Even though I do a fair amount of Sibelius, I don't have one in my music room -- I only have a real piano. You also need to make sure your setup has the MIDI keyboard right next to your computer, which isn't super convenient.
It's true, I'm not a professional in the sense that you mean. But I am a serious musician doing real work (some local groups perform my work, and I work on editions of old music too). If it's well executed, I'd much prefer working this way to the existing options.
I like to write music away from a desktop/laptop computer or keyboard, and this app seems to be a significant improvement over apps like Notion for that purpose. So I would say that it definitely does approach a problem in a new way, with potential to get closer to solving the workflow than any other mobile app thus far. I don't think it is fair to compare it to a MIDI keyboard workflow which is really a different environment than what this app would be useful for.
If you want to explicitly enter 16th notes, you have to make it obvious to StaffPad that that's what you want, otherwise it can interpret the lines as 8th or 32nd notes, or a combination of those depending on where the notes fall in the measure. Again, for simple scores where the handwritten music is fairly obvious, it works fine. The key point is that the entry method slows down professional users, because you have to make sure you're doing things that the program can understand, which means taking time to place your marks methodically and reviewing every entry you make to make sure the program was correct to your intentions.
it looks like you'd add dashes for the tails, just like you would for a semiquaver on paper.
one of the great things about this is that there's zero 'design' to sheet music, its essentially a spec.
There's a lot of implied information that can be fed into the handwriting recognition, and it looks like it doesn't try to parse until you've moved onto the next bar, so can analyze the block against a very well defined domain.
Vertical lines all represent a note (besides small ticks for sharps maybe), and the time signature narrows this down to only so many possible permutations of possibilities.
I have a bit of experience with transcription/engraving software. I completely agree that the process tends to be slow and frustrating, but I'm not sure if this really solves the problems. This looks like it could be nice for rapid prototyping, but I suspect it would be at least as equally frustrating if you're trying to get a professional end result where you have absolute control over everything.
My favorite engraving software is LilyPond, which is similar to LaTeX in that it's just a plain text language that gets rendered. There is no GUI, although there are several "IDEs" that I have tried (they seem to be mostly useless beyond syntax highlighting, templates, and a live preview). And like LaTeX, it is often extremely frustrating, and the learning curve is extremely steep. But I think I prefer it because, once you get some experience using it, it is (mostly) very predictable and gives very fine-grained control.
> For those that haven't written music before, previous programs are some of the clunkiest, slowest, most infuriating pieces of crap that I've ever had to deal with.
Well, first of all - this is not the app to write music. It's the app to write music score. Although they brag about playback with something akin to a sample-based orchestra, there's not even a hint on traditional DAW capabilities, so you'll have to export the result midi into your DAW of choice and then get into the the whole music-writing process other than arranging anyway.
And on other hand, have you tried the latest Logic? It seems pretty great when it comes to the score interface. Unlike this app, it's also one of the best DAWs out there, and has an enormous default sound library.
never typed music... but i can't bring myself to believe there is no easy way to enter a note with 104 keys keyboard right in front of you... is it really that bad?
i would imagine one keystroke per note. maybe one modifier for things like several octaves out.
The problem is the combination of note length and the actual note. Add in multiple notes played at once and you hit the 3 key limit of most consumer keyboards before most groups would be finished.
Beyond that there's also just the difficulty of mapping between the keyboard and keys, how do you easily deal with the gaps in the black keys, there's enough keys to map one to one with most of a piano but keeping that in memory while notating music is a big ask.
Sure, but that's not what the parent I was responding to was suggesting. gbc0 was specifically talking about using a 104 key keyboard to write out music, so I addressed that option and it's pitfalls. Other people have addressed issues with the midi keyboard option though.
I can't really address that very well because it's beyond my experience, but I'll summarize what I've seen. The main gripes seem to be it's still a little slow to enter notes because you have to specify the note length each time you want to change it and that it takes up a lot of space compared to something like this where you can just write it out when it may or may not be your instrument.
Interestingly, it interprets each pen stroke in the context of the wider piece of music, and makes use of temporal information such as the order the strokes are made in. Whereas classic OCR would just try and interpret the static 'picture':
Key to the way StaffPad works is its method of recognizing your scribbles. It looks at every individual stroke you make and then interprets what you wrote based on the relationship of each stroke to all of the others. David says that “it’s more efficient and accurate to take the position and temporal information from the pen, and then use musical context to decide what the music is trying to be. That way, you can do things that would totally confuse OCR. Because we know the order of the strokes and where they are in relation to the notes, we can say, OK, that’s a natural, that’s a sharp.”
Staffpad is incredible but it's not brand new or developed by Microsoft. Hanselminutes (Scott Hanselman) did an excellent interview with the engineer who created it a few months back, definitely worth listening to if this piqued your interest. Thoroughly covers how it was conceived and coded in C#/C++, with comments about the relationship to Microsoft: http://www.hanselminutes.com/473/developing-staffpad-a-new-c...
It's actually a project started and run by my brother! I'm tremendously proud of him and what he's achieved with StaffPad, whilst simultaneously running a business as a contract music producer as well no less.
Seeing him on stage with Nadella was a particular highlight.
As far as I'm concerned, this is world-changing technology - a (seemingly?) easy to use, natural-looking, efficient data-entry mechanism that looks like something from the future.
That is an absolutely fantastic demo - I was hooked on the background-video within 3 seconds, and found myself wishing the MP3 would play when he was exporting it at the end.
In light of this example, it's interesting to consider that world-class pen spinners [1] might have opportunities as advertising consultants. "Let us perform or teach you the exact pen spin that's required for your advertisement - should it communicate confidence, sophistication, should it be quick n' simple or slightly over-the-top? We've got you covered."
I've been working on an iPad app in a similar direction, though with a different end goal. As I'm sure the founders of this company can relate, I've found Sibelius and Finale incredibly frustrating to use, especially for anything resembling experimentation. However, I also feel that sheet music is generally incompatible with modern popular music. (See the Aphex Twin joke below!) Much of my favorite rock and electronic music has syncopation, changing meters, pitch bending, and a general fluidity about it that's incompatible with the rigidity of staff notation. Plus, staff notation is very difficult to read.
With my project, I'm hoping to bring some of that flexibility back into the compositional process. In short, it's an infinite canvas with time on the x-axis and pitch on the y-axis that you can simply paint with your fingers. You're not constrained by key signature, time signature, or note length, though there are snapping tools to help with that if you need it. It's meant more as a tool to take musical "notes" than as something that can produce a clean final product. As someone who doesn't play the guitar very well (yet), I'm looking forward to using it to experiment with guitar solos.
I'd like to wish you the best in your project, and mention that as a huge fan of modern drum & bass and other EDM forms, plus as a longtime guitar player, I believe you're actually working in a space that lacks such a tool. The video makes me smile. That's a nifty pocket resource to me!
Just going to spit-ball a couple thoughts based on my experiences and your write-up!
I've been using Blend.io to share my Ableton Live projects with others online (via Dropbox). This is helpful because settings are exported - do you think that in time you might have a mechanism in the program to list & chart out the settings for a synth, say like Massive? Could be a whole can of worms, but important stuff to recall going forward when taking notes.
As for guitar solos, I think you could use one of the blues Kings for material in testing, either B.B. King or Freddie King, depending on how much pitch bending or note rapidity you'd be interested in. If you want to get extreme, Dimebag Darrell of Pantera had some crazy harmonic & Floyd Rose "dive-bomb" tricks he'd use that might test capability!
The initial offering will be pretty sandboxed from the rest of the production toolchain; I'm aiming for an MVP with the simple goal of "sketch music quickly". So there won't be much to offer in terms of interacting with other software.
After that, however, anything is up for grabs. Eventually, I'd love to add features that would make it an invaluable music production tool, not just a note-taking app. Under the hood, each instrumental layer in my app has a set of "traits" (volume, vibrato, expression, etc.) that get matched to the appropriate settings in the output instrument if they exist. Only the settings that exist in the output instrument are used; the rest are ignored. For the time being, these traits are only used internally for the built-in MIDI synth. But if I'm understanding you correctly, this means that it should be pretty simple to add arbitrary traits for outside instruments. And eventually, when I add Audiobus and/or MIDI-out support, it should be possible to play the track directly through the synth! (But that's a bit into the future... need to focus! Need to focus!)
Thanks for the guitar solo tips, I will definitely be using some of those!
When I saw this my first thought was: We need something like that for Math / technical drawing.
I can understand that music is a good first application of this technology, but entering mathematical and engineering-related content through such an interface could be a huge deal as well.
OneNote does this. You can either insert a new equation and get an editor, or select existing strokes to convert (to either writing or math). There's sometimes error correcting involved, but it's pretty impressive.
Man absolutely, right now I'm filling up a blocknote really fast working through problem sets, trying to minimise paper by writing small, fitting many problems on one page etc. Would be lovely to use a digital pad and save each problem set on its own page and have an easy eraser tool.
Anyone used the Khan Academy app to brush up on some math? They've got a very nice handwriting to text feature to allow easy entry of answers on a mobile device, a pen would work great in combination.
I've been using Xournal for notes, problem sets, etc. for the last 5 years. It doesn't do handwriting recognition, etc., but I find that this is a good thing -- that way I don't ever have to go back and fix recognition errors, and unless you are preparing something for publication, typesetting math is a waste of time.
It's actually possible to LaTeX as fast as someone is narrating math symbols. Definitely not as fast as writing on a whiteboard, but on the same par as typing English.
Microsoft's iPad Pro demo showed drawing geometric shapes in PowerPoint. Reminded me of drawing in Flash but not quite as free form. I'd be very surprised if PowerPoint on Surfaces never got similar features. Maybe it does already? I don't have Office on mine.
I think math would require more error correction than music. Musical notation is pretty easy to get right, whereas math has a much larger symbol lexicon.
Musical notation can actually get really hairy even when restricting yourself to pre-20th century conventions. Another notation product currently in development has a very interesting blog that delves into some of these issues (http://blog.steinberg.net/ ).
Math, on the other hand, is actually way easier even including things like mapping diagrams.
FWIW, Brown University was doing research in this *Pad field (stylus-based input apps for domain specific notations) in the late 90s/2000s. I was a member of this group for a brief period of time. It's really exciting to see a resurgence in high quality pen computing with affordable, entry-level consumer hardware like the Surface 3.
Curiosity: For all those ethusing about this, do you have keyboard skills? Piano was my primary instrument and my take on this may be skewed by the fact my primary instrument also turned out to be pretty good for composing music with, when you can literally bash out full chords in real time if the mood strikes you. If your instruments were all single-note instruments that don't hook up to computers worth a darn I could see this being much more exciting.
When you write music with an instrument, often your ideas take on the idioms of that instrument, and the music you write is limited by your technique at the instrument. That's why I encourage composers to try writing without being at the instrument, as it gives you more freedom once you get used to it. This app appears to offer a compelling entry into that kind of workflow.
Composition by writing out rhythms and chords, and writing out multiple parts for multiple instruments, is just a different experience than playing it out on a keyboard. It lets you put more thought into how things line up and are organized, versus how they sound. That sounds downright unmusical but it's a big difference, I think.
This is just from my limited experience as a hobby composer sitting in front of a keyboard with pencil and staff paper, but I've heard it from other composers I know as well.
I did it with synthesizer and computer. What you say is still true, though, since that probably qualifies as yet a third way of composing.
For that matter I instantly noticed a difference going from ScreamTracker to a MIDI synth, too. Didn't even have an acclimation period, it was immediately of a different nature.
Some composers (especially those with perfect pitch) can write out music without an instrument. Mozart, Bach, etc.
Many composers that use a piano will also write out the notation by hand without a MIDI interface. In the scene[1] from Amadeus, replace Salieri's quill and paper with a stylus and the iPad/Surface.
I can see Staffpad software being useful for both types of composers.
Yes, I do but I'm stubborn in my current production process within Ableton Live Suite, but I have my reasons. As in I record single takes as audio files, not record MIDI inputs that would drive a synth engine. Well, I do that for drums but I finger drum naturally then build the drum line from clips/loops of my own usually with minimal corrections or quantizing.
Honestly I'd love to use this thing for Guitar Tab when sitting down and figuring out songs / covers. I still use pen and paper for that. Again, Ableton Live Suite has powerful tools to help - converting music to MIDI, both rhythms or melodies - but I do my own thing.
This amuses me, because they used the video at the top of the linked site in the presentation... which shows a Surface Pro 3 and pen (and was filmed ages ago).
SP3 is great, obviously, but it was in the middle of boasting about the _new_ pen and how great it was.
Just wondering - wouldn't it be easier, not only for music notation but also for math and other scientific notation, if there was an on-screen keyboard custom-built for this very purpose with the required symbols and movement keys? After all, why require a stylus and why sketch the characters when you can just as easily type (actually, typing may turn out to be a much faster input method). Is there a real benefit to sketching or is it something else.
Notice that "keypad" in the bottom right corner. That's the "on-screen custom keyboard" you're talking about.
But this isn't very convenient:
1. There are a lot of symbols. So many that the keyboard is divided up into six sub-keyboards. Unless you know the program well, it's a bit of a mystery where exactly to find the right symbol.
2. And it's actually more complicated than that because some of the symbols are actually modifiers of other symbols. For several of the symbols you see (sharp, natural, flat, dot, tie) you can't actually draw them independently, they only exist as attachments to another note. So the keypad works differently when you are dealing with those symbols.
3. Plenty of marks aren't even covered by the keypad and its six panels. For those you have to go searching in the numerous other menus in the app. You start getting a Microsoft Word-like problem: how do you design menu bars that show all the options without taking up the whole screen.
I really think it would be easier to draw what you mean -- if the app is actually good at recognizing what you mean.
Speaking from a music composition point of view, the ability to simply write a note on "paper" is huge. There's a big difference between sitting at a piano with a tablet that you write on and a laptop with a keyboard and mouse. Even with a custom "music keyboard" getting the nuance of written music through digital input is slow, difficult and frustrating. For years we've been able to hook a keyboard into composition software, but writing in dynamics and other notation meant clicking through menus and sub-menus in crowded, tedious interfaces.
The ability to write as you think/feel it and have the computer handle digitization is a big step, and hopefully allows composers to just write, without working around their software's limitations.
What you describe is an app called Notion for iOS, and it's pretty bad. I'd take pen and paper over Notion any time. This app definitely approaches the pen/paper situation much better. It doesn't try to be a MIDI music app, it tries to improve on the process of just using your handwriting, which I think is a better fit for a tablet.
Music is hard because there's simultaneous linear and horizontal organisation. So picking symbols and placing them can be harder than it looks.
I'd like to see this app being used for something simple-ish like the Prelude in C from Bach's WTC. The usual notation shows three independent lines - two in the left hand - and I don't know if the app has enough awareness of context to make that easy.
I haven't used Sibelius or Finale for a while, but last time I looked you had to enter the lines separately. You couldn't just "write" them in one pass and make them fit together, as you can on paper.
As for handwriting - I don't think anyone really wants it. QWERTY is faster, easier to read, and more accurate.
Symbol input (math, music, etc) is different. But for basic text, I don't think there's any clear advantage to using handwriting for input.
Experienced composers and arrangers will have spent years writing on staff paper, building speed and accuracy. This shorthand and muscle memory will be irreplaceable for their target market.
You get a sense of the experience watching the video: quarter notes are just a short diagonal line, all the note stems are perfectly vertical.
In a way, this problem space is perfect fodder for automation. If a musician can sightread off of your chicken-scratched sheet music, the computer will be able to do a decent job too.
Being able to select a note value from a palette, and then just drawing the head without having to bother with stems and beams would be a time saver in this case.
So, the MyScript team (behind the Stylus beta handwriting tool on Android) released the MyScript calculator[1] a while back that lets you start doing (albeit relatively basic) math with the stylus on a touchpad. (I use it on my Samsung Note Pro.) It has a similar feel / intent to this thing, but somewhat less sophisticated (mind, it's evidently a proof of concept demo more than anything else).
The StaffPad app was reviewed at least 5 months ago [2] and truly does look very impressive, even if it's of particular interest for a relatively small demographic. I noted there was no mention in the SP4 announcement of the USB / sound device latency for the SP4 with W10 - so it's either so good we don't need to mention it, or a bit uncomfortable so best not to mention it.
Using generic ASIO4ALL drivers I'm able to get latency around 30ms using "generic" sound cards like the stuff onboard a ThinkPad. ASIO4ALL takes over the sound device so only one app can use it. Without it, latency can be rather much higher.
Interesting - to me 30 ms is pretty bad. I've got a ThinkPad i5 with 8GB running Win10 and can get in the 10-15ms range in Ableton Live Suite 9, and I have access to a SP3 that I might be able to test out. Granted, I don't think the design of the software is for "live" type use but I have noticed there are some 'sweet spots' for certain programs in the buffer size, e.g. Kontact doesn't like lower than 128 for some reason, whereas Traktor Pro doesn't care in the least.
Hello! I make Soundslice (soundslice.com), which is interactive/reflowable sheet music in the browser, synced with audio recordings and more loveliness. I very deliberately use canvas instead of SVG.
Soundslice isn't a score editor -- "just" a renderer (as if rendering sheet music is easy...!) -- so that influenced my decision. But if I were making an editor, I might make the same decision.
The reason is performance. A single piece of sheet music (a "score") might include thousands of individual glyphs, and if each one of those is a DOM element (as with SVG), the performance just wouldn't be as good as canvas.
One big tradeoff is that, with canvas, you need to implement your own hit detection (e.g., figure out what the user clicked on). But that's not super hard, and it's the kind of thing you write once and use forever.
What would be the advantages of using one over the other. I'm in the process of refactoring/improving a web game (connwars.com) I made a while ago and I was planning on using canvas, but I don't know if svg would work better.
Use canvas if performance is important. Use SVG if it's important to assign functionality to specific graphical elements (as opposed to rolling your own hit detection). See my comment above in this thread for more.
[Usual caveats apply...this is general advice, and I don't know your situation, etc.]
I don't think you'd want to try and do this with web technologies. I followed StaffPad's development pretty closely (due to it being a project of my brother). They pushed what even Windows native apps could do to the limit to make this. It's performance sensitive and benefits a lot from serious pen APIs. The web is not a reasonable platform for solving every problem and this sort of boundary-pushing productivity app would be an especially bad fit for HTML5.
Now that Apple Pencil exists, I hope they consider a port. This looks amazing but I just don't think any app could convince me to buy a Windows machine.
"In the Periscope, Ranjo was impressed with the grip of the iPad Pro's screen when used with the Pencil, and he also was a fan of the palm rejection features. When drawing on an iPad Pro, the palm of a hand can be rested on the screen, but it's still responsive to zoom gestures."
I figured as much; I've never thought Aphex Twin's lines were particularly challenging, he's just got a very unique approach to sound design and composition.
Would you prefer a technically playable piece by way of Michael Angelo Batio?
I would love to see a programming/math prototyping environment based on this idea. Transforming data by sketching. Quickly writing down a formula and pulling data through it with an arrow. Putting filters on the arrow. Like Wolfram Mathematica on steroids.
The idea of writing via a stylus and having it transformed not into a massive PNG, but perhaps a simple Unicode file is really growing on me.
I love pen and paper for jotting down ideas or using it to solve problems, but it seems this medium is slowly going to take over due to the benefits of having a software tool analyzing your writing, and thus providing tools/suggestions/etc. along the way.
I really want to try it out of curiosity, but as someone who doesn't actually know how to read, write, or play music I can't justify actually paying for it.
All the more reason to get it! Unlike other tools requiring more intermediary technical details, this lets you get straight from note-on-staff (in the most natural way possible short of actual instruments) to proper notation to hearing it instantly.
I'm actually considering using Windows because of this and the new hardware. If you had told me that two days ago I and my beautiful Arch setup would have laughed in your face. Kudos to Microsoft/Staffpad, this is really great.