Minor bug, Beethoven's 5th was off by a measure for me. (Goldberg variation was spot-on).
Also, right/left arrow keys aren't skipping measures (as the keyboard page says), but are skipping whole sets of measures.
But overall, as a musician, I actually found it rather distracting to watch the line try to follow the notes exactly, as the line constantly sped up and slowed down due to rubato, etc. -- indeed, in particularly fluid pieces, it can even be possible that the right and left hands don't match up precisely in actual performance.
It might actually be more clear, and far less distracting, if you followed the time signature for the beat, and simply highlighted all notes in the current beat within a light-pink rectangle overlay, and then that box would instantly disappear, and the next one appear, upon the next beat.
That way, we can focus with our ears on the actual movement in the music, without being distracted by visual movement, but still have a perfect visual indicator of where we are.
But anyways, great work and great idea! Kudos, well-done!
Oops! I forgot to change the definition of the keyboard controls... it's actually meant to turn a whole page so I just need to change the definition.
(just moving by one measure is more easily done by clicking in the score...)
Thanks for correcting that :)
As for Beethoven's 5th, it should self-correct, but I have noticed that in some browsers it doesn't quite work... What browser and hardware are you using? I will look into this further.
I have heard from several others that the current way of moving the cursor isn't optimal. Your solution could work. I'm going to explore a few ways and probably offer some new choices to the user. Your suggested solution would be very easy to implement, so most likely I will add that.
Tried it again and the Beethoven's now fine. (Chrome, OSX.) The Beethoven audio file had taken about a minute to load though, maybe it had to do with that. Anyways, probably just an edge case.
One minor "bug" / missing feature; I have two monitors at different resolution; when moving the browser (maximised) from my laptop monitor (1440 x 900) to second display (1280 x 1024) the staves are put on a different line to the instrument names. If I refresh the browser on my second monitor then move to my laptop screen all works fine though.
The same issue exists if I resize my browser; i.e. currently window.onresize events aren't handled.
Great work though - and nice choice of pieces too.
I like this. I’m not as impressed with it as many others seem to be, since I’ve worked with Encore and Sibelius a lot and they have almost this exact feature. That is, they play synthesized audio while moving the bar along with the sheet music. But it is nice to have better-played music to listen to while following along with sheet music.
I do also sometimes pull up some classical sheet music and try to read it along with a recording. So if you get a bigger library that includes pieces I end up being interested in, your site would be useful in that respect. I would enjoy the safety of having the red bar following along to make sure I don’t lose my place.
You said on https://news.ycombinator.com/item?id=6226990 that the left and right arrow keys should move by page, and not by measure, and they’re just mislabeled. I think it would actually be more helpful for the arrow keys to move by measure. That’s what I’m used to in other programs, and I would want to do that more often. I often move back a measure or three back to replay a complicated part I just heard. Moving back a whole page is too much, and using the mouse to click the previous measure is less efficient. You could still keep the keyboard page-moving functions, but bind them to Page Up and Page Down instead.
I’ve used Transcribe! (http://www.seventhstring.com/xscribe/overview.html) before to manually mark up the measures and beats of pieces. You say you have some JavaScript tool that helps you mark up pieces, and I would love to see it, but you might also want to look at Transcribe! and see if that makes marking up pieces easier. Transcribe! uses a proprietary file format, but it’s not too hard to extract the beat times by parsing the text of the file.
Another scrolling method you could try to help people keep their place is showing two rows/systems at once. When you get to the end (right side) of one row, move the bar to the left side of the next row, and then scroll the next row up, displaying the row after that below it. I’m thinking of the effect in this video, which uses Finale Notepad: http://www.youtube.com/watch?v=CxTav4S-4Tk.
I'm also a long-time Sibelius user, but was missing high-quality (real) audio together with the playback... So thanks for the comment.
The javascript that I use to partition a score is extremely simple - all it does is play the song while I click on the note in the score that is currently being played. This allows the script to "register" the x-coordinate together with the the current position in the audio, at each click.
That results in an array of [x-coordinate, time] pairs, which are then used to control the speed of the cursor movement.
If it makes sense for to let people upload their own pieces (some have asked for that), then that functionality would certainly be made available!
As for the scrolling, I think that it's great for piano pieces (or pieces with just a few parts), and I might consider implementing that also. It wouldn't be possible for larger scores with multiple instruments, but I think it would work better than the current solution for smaller scores.
Finally, great idea regarding the measures / page movements! I will re-map the keys so that page up / down scrolls a whole page, and the left right keys just scroll one measure at at a time. Please look out for that improvement in the next day or two ;)
I really like this and would use it at a concert, but the page turning is very jarring. Having the time sig fixed while the sheet scrolls like some sort of symphony hero would be pleasant.
Hell, you could even color code the instruments subtly with some light shading. Maybe make that a checkbox option.
Well it seems that consensus is that the current page turning is not optimal... Perhaps scrolling is the solution. I will definitely invest some time in that to improve it.
Also, regarding the time and key signatures, I very much agree with you, they need to "stick"...
As for color coding, is the purpose to make it easier to follow a certain instrument?
As for the page turning - I think you could alleviate a lot of the issues by just slowing it down somehow. I can't remember if it was animated, but, if not, I think it would be less jarring if there was an animated scroll effect, starting slowly and then getting very fast, to cue the viewer that we're about to change pages. Similar to how the iPhone transitions screens, but maybe a bit slower since with the iPhone the user initiates the action and is therefore more prepared, perceptually.
Got it. It is is currently scrolling (but very fast; I think I set it to 100ms for every page turn), so I could certainly slow that down. I will experiment a little bit and see how it looks. Thanks for the suggestion!
The colors would be both aesthetically pleasant (i'd go with pastels) and make it immediately obvious what you're looking at. You could also put the labels next to the note chart on the left.
I think highlighting a particular instrument or two (that you are either playing or focusing on) is a good idea and could be implemented as a checkbox option... so thanks for that suggestion.
I like it. I have a few suggestions (full disclosure: I haven't played music since high school).
It would be nice if the key and time signature could remain onscreen as the sheet scrolls.
I think my preferred solution for scrolling would be to have the position marker move until it hits the center, then have the position stay centered while the sheet scrolls until it hits the end. I'm not sure what the correct way to handle repeats in my scenario would be--perhaps have the sheet stop scrolling when it hits a repeat.
I think having the key / time signature "stick" makes a lot of sense... I will work on that. The only thing to look out for is key signature / time signature changes, and make sure those get captured too and that they stick...
As for scrolling, it seems like that is a feature that some people might prefer. I will most likely add that as a feature to it.
I think having the page stay still for the majority of the time (like it is now) is better than to have it constantly moving, but when you do change pages, scroll to the new position (ease-in, maybe ease-out too).
Removes the disorienting jolt of having everything just change, but also not always moving and being a distraction
Maybe additionally highlight the notes that are currently played. And have you tried scrolling the sheet instead of moving the 'now' indicator? And who are you and why are you doing this? Add an about section.
Yup, I can do the highlighting fairly easily, that's a good idea!
Scrolling is easily doable too, but I'm not sure that's the best way for all situations. I might provide it though, and then let the user choose depending on their situation.
About me: a wannabe classical composer / tech geek ;)
(and in the event people are actually interested in this idea becoming more than just a concept, I would be very happy to provide a more elaborate About section)
Even a "right now this is just a little concept I worked on in my free time, hope you find it useful. Tweet me suggestions @..." would be enough on /about.
It's a demo, but you may find some UI ideas interesting, for example synchronization and navigation through a video, flowing score, hiding the instrument parts that are not playing for a while, and assembling a score from small pieces with measure images hosted on multiple subdomains to improve the fetching performance in the browser.
I have considered that, actually. There are certain advantages with it - although, one thing that I like in particular with hosting the audio myself is that for example, I can slow it down to 25% speed and still have it sound great! Not possible with Youtube at the moment (if you slow down their videos, the results are deplorable).
Interesting demo! I definitely like the hiding of the instruments not playing.... also, lots of people seem to prefer a flowing score.
As for fetching more quickly, yup, also worth addressing.
Hey Adrian, I accidentally discovered your site just a few days ago and I think it's great! Neat solution to use Youtube to pre-empt copyright problems as well.
I'll send you a tweet or email so we can stay in touch!
Wow, that's the best I've felt all day. Truly lost myself in the piece trying to follow the red line and tease the individual notes out of my perception.
I will definitely work on getting that line synced up. On my computer (Mac OS X / Chrome), it's perfectly synced, but it seems that that isn't the case for some people.
FYI, I'm on a 2011 Macbook Air running Safari. It's good enough that I enjoyed the experience. Honestly, I was never absolutely certain that the red line was off. Just suspicious. :)
Listening to classical pieces with the score was one of my favourite things to do when I was at music school. One thing that would be cool would be an overlay of the clef, time signature and current (declared hah) key on the left hand side that updated as everything changed. Good for context as you're following along.
That terrifying moment when you think that your project is on the front page [http://notezil.la ] when it's really a completly unrelated website with the same name >.<
noteZilla's well designed btw. It's also beautiful as an observer, but, if I were trying to use it as a performer I'd be thrown off by the varying placeholder speed and page turns [as others mentioned]... maybe that should be your toggle if you want to accomodate that auidence — a 'performance mode' which shows multiple lines and a consistent rhythm with highlights around the active bar.
And, I think I will share even though I'm ashamed atm.
FYI, I slowed the page turns down a little bit... in an effort not to jolt users too much (a first step). If you're reading this, maybe check the site and see if it's any better, and let me know?
Also, thinking about implementing multiple lines as well (just haven't decided how high up it will be in the list of priorities!)
Check out http://vexflow.com It's got a lot of interesting features and the code is really usable... it has tab support for the guitarists (like me) who can't read music, it's open source, html/svg rendering, and it's got on a pretty interesting text based DSL for authoring music. I even wrote a plugin to integrate VexTab on Mediawiki but I haven't done much with it after the proof of concept. Too busy!
I have been wanting to work on this idea for quite a while in my spare time, particularly to enable scrolling and keeping the signatures in place while the sheet scrolls.
As a guy who is learning the piano but doesn't know how to read sheet music very well, I'm really excited about this as a learning tool. Would it be possible to show the letter notes next to the notes on the sheet music? Also, how easy is it to incorporate new pieces... could I potentially submit sheet music that I have purchased and you could convert it for me for a fee?
Yes, I am exploring adding more functionality to it, and one thing that I'd like myself (and I think, you too), is piano tutoring.
For example, it could play a certain section once, then repeat it, but without sound, while the student plays, and then continue in a loop like that until the student feels comfortable with that particular section.
I'm sure there are many other things that could be done in that realm too, so feel to tweet or email me anything that comes to mind (@notezillaio / notezilla.io@gmail.com)
As for incorporating new pieces, it's definitely technically doable and it seems that that's something people are interested in. There might be certain copyrights issues that have to be ironed out first though... ;)
If you're wanting to get better at reading sheet music you really should not have notes being shown. It will be harder to read but that just means you need to start simpler and build up over time. If you get used to reading notes it will always be a crutch you have to work with, and instead of sight-reading the notes themselves you'll be more accustomed to sight-reading the letters.
You might be right about that. I still find though that the most daunting part about learning something new for me is grinding through writing the notes in on the sheet music then "sounding it out" going back and forth from my computer/ipod/ipad getting the timing right. This tool seems to wrap it all up into one neat learning process, perhaps one day after using it enough with "notes on" I could uncheck that box and go pure sight reading.
Really nice. Now I know the meaning of those squiggly lines. I also like the sparse full screen layout.
A few minor things: While jumping to time points works, scrubbing does not. Instead, the notes image gets dragged. You could embed the image as a background-image instead of an <img> to prevent that. Or catch the dragstart event on it and return false (http://stackoverflow.com/questions/4211909/disable-dragging-...). A custom Rails error page or better error handling would also be nice (http://notezilla.io/bla). And there are a few console.log's left (TIME!!!! :)
Thanks for pointing that out, I didn't anticipate this level of interest so I suppose I forgot to take out the console.log stuff. Will do now. Glad it wasn't anything embarrassing ;)
As for scrubbing, it was a conscious decision to keep it fixed for now, but I might consider changing that in future version... and also need to fix status pages, etc.
Yes, I match it by hand, but it's fairly easy to do using some javascript that I wrote...
I agree, this is super cool. Really simple and elegant. For something along the same lines in IOS app form, I found UK developer Touch Press' The Orchestra does something similar but also mixes with a video feed of the Orchestra itself, and a running commentary http://www.touchpress.com/titles/orchestra/
This looks prettier than the MuseScore web interface.
Please tell me you will have some way to use this to display a score written in some open format. and/or interface with MuseScore or some other software.
Despite having a reasonably complete Unicode loadout on this Linux machine (that is, yes I've gone through my distro and binged on the fonts, yum yum yum), I've got a lot of boxes with hex in them. The entire what-I-presume is a control bar at the bottom is hex, and there's two things in the upper right that are hex too. You may want to consider adding a web font for those things, if you can, though I haven't got a great quick way for you to test that that is working.
Strange. I'm using Font Awesome for the icons, and I was assuming that they would be working just fine on Linux machines. I will see if I can replicate your error on my own Linux distro. What setup are you using?
I prefer the entrancing visualization done by this guy: http://www.youtube.com/watch?v=rRgXUFnfKIY (his YouTube channel has much more). As a semi-trained musicians, I do see value in showing the full score, but purely as an audiovisual experience I think removing some of that information and just using colors for instruments and y-axis for relative pitch is more mesmerizing.
In music sequencer a.k.a. DAW software (Logic, SONAR, Cubase, etc.) this is usually called "piano roll" notation. Because it's essentially a player piano roll -- you know, the original lossy compression. ;)
Such software usually offer 3 views of the same underlying music data: piano roll, staff/sheet notation, and an event list.
It's mostly manual work at this point, although it's not too bad because of some javascript that I wrote that makes it really straightforward to partition a score into time intervals...
And yes, I might actually write a blog post about this so make sure you have signed up on the site :)
First off, really nice job. Secondly, if you're not aware, audacity has a segment labeling feature - (I'm just throwing this out here because it might be easier than what you're doing). Select time boundaries and [ctrl+b] will label it. Labels can snap to abut each other. Each label might represent the span of a beat. The labels can be exported as an easy to parse .txt file and can be used to determine cursor position as a function of time.
Sonic Visualiser [1] can---or, at least, it makes an attempt to.
It's a highly useful tool for the analysis of music recordings.
I, too, am highly interested in the extent to which matching recordings to scores can be automated, as are a lot of music theorists. I hope you can publish a write-up of the process for syncing the audio and the score.
If there is a significant human spend required, then developing an open tool for community contributions would greatly speed up the creation of a corpus of scores.
I think a good case could be made for such an automatic system... although it's probably a somewhat too complex / labor intensive task at this particular stage for this particular project!
As for the write-up however, the process I currently use is very simple (and it's pretty much all "manual labor").
To partition a score into time intervals, I have a couple of lines of javascript that record the time the audio is at, together with where in the score I click (and where I aim to click on the note currently being played).
It doesn't take too much time and effort to partition a score this way actually, unless the piece is slightly complex. Beethoven's 5th took a little more effort than Air for example ;)
Hope this helps for now! And I am planning to publish a more detailed write up of everything, so please stay expect that in the next month or so (easiest to hear about via Twitter or by submitting your email to the site).
I wouldn't recommend using this structure directly since it requires the project to have a specific structure. There are some scheme callbacks in the writeSpaceTimeDumper function which you can use directly to extract the information that you can use.
I love (classical) music and as a music hobbyist I try to dig more into sheet music for a while. I would also pay for some kind of premium service (maybe great content or listening the sheet music in combination with a famous record).
Anyway, I subscribed to your mailing list, please just keep on rocking.
PS: And yes, I own Logic and could play sheet music in that way. But I do think you folks will prepare something more which a hobbyist can enjoy!
I think so too :) Thanks for the subscription and the nice words! I definitely want to hear your ideas, so please feel free to get in touch (contact information listed under http://notezilla.io/library)
I would be happy to release parts of it as open source, and also very happy to consider implementing functionality that allows teachers to - for example - create lessons for their students.
How about you email me at notezilla.io@gmail.com and we can chat more? Would love to learn more about your thoughts and how this could be useful to you.
What I was missing is some indicator showing where exactly in the whole piece I currently am. A simple solution would of course be a timeline but maybe something neat like a minimap with a sliding windows of the current view is possible. Like this one could directly navigate through the whole piece by clicking in the minimap.
Thanks for a great suggestion; that's a feature very likely to come in the near future actually! I'll probably post it together with a few other updates to Twitter, so please stay tuned!
Cool project!
Just found this link (are you aware of this?):
http://www.ee.columbia.edu/ln/rosa/matlab/alignmidi/
They seem to have automated the process.
Apparently they synthesize the midi, then somehow align the beats.
Wow this is so great. As a musician listening to a classical piece (well and an engineer too) I often think, "How does this work? I hear a clarinet here, is that what it is or something else?" So yeah, very cool.
A bug: If it reaches the end, then clicking in the score will move the pointer there and start scrubbing but no sound is played. The play/pause button behavior becomes a bit funky as well.
I love this so much! The possibilities of the Internet with art are just endless; this is just one more example. Great work. Looking forward to where this goes from here. I'll be watching! :)
Most likely that will be very hard to do, at least for orchestral pieces, which are typically recorded using overhead microphones (ie, not with separate tracks for each instrument).
For pop music, it's definitely doable as long as I can get access to the individual tracks. It's actually something I have been wanting to do, so thanks for the suggestion.
I'm actually using LilyPond already to create the sheet music, so if I do add the possibility for users to upload their own scores, it would most likely be with LilyPond...
Although the user would still need to partition the score (ie indicate at what time a certain note occurs) if the music was to be synced up to it.
It isn't terribly labor intensive but still requires a little bit of work... so maybe, if that's what people want!
Oh hey, sorry, just realized that you are actually the original author. See my comment above suggesting MusicXML instead, since it's more widely used and machine-readable. As for specifying times, you can use the note lengths and tempo markings to generate a rough timing and then allow the user to tweak it to fit the music.
But this is awesome by the way, great job! I'd really love it if user-generated content is supported. I have a bunch of lilypond transcriptions from the Real Book that I'd like to upload.
Cool, let's stay in touch because I also think that user-generated content is the way forward, and I agree about MusicXML.
Just need to be clever about it and make sure it's not too arduous a task for the user to upload something. Please feel free to email me or tweet at notezilla.io@gmail.com / @notezillaio and we can stay in touch / discuss further, would love more input!
Actually, now that I think about it, it might be better to support MusicXML, which is more machine-readable (you just need an XML parser). Lilypond and many other music notation systems like Sibelius and Finale can export to it.
Thanks for the heads up, I will fix it this weekend along with a few other things - I didn't anticipate this level of interest so there were a few details (including that one) that I had overlooked ;)
Some of Sibelius's core developers we let go recently. Steinberg hired them and they're writing a new music notation application from scratch. They're still in the very early stages of development. (see http://blog.steinberg.net/)
Full of bugs, only canvas draws correctly on mozilla. But I admire how you can get a product online like that and I can't because I'm a perfectionnista.
I'm using metrica.yandex.com to study user behavior (mouse movements, etc) - extremely useful and free, and in my opinion far more telling than Google Analytics!
I can highly recommend it. You need to activate their WebVisor functionality to get access to what I just mentioned.
Minor bug, Beethoven's 5th was off by a measure for me. (Goldberg variation was spot-on).
Also, right/left arrow keys aren't skipping measures (as the keyboard page says), but are skipping whole sets of measures.
But overall, as a musician, I actually found it rather distracting to watch the line try to follow the notes exactly, as the line constantly sped up and slowed down due to rubato, etc. -- indeed, in particularly fluid pieces, it can even be possible that the right and left hands don't match up precisely in actual performance.
It might actually be more clear, and far less distracting, if you followed the time signature for the beat, and simply highlighted all notes in the current beat within a light-pink rectangle overlay, and then that box would instantly disappear, and the next one appear, upon the next beat.
That way, we can focus with our ears on the actual movement in the music, without being distracted by visual movement, but still have a perfect visual indicator of where we are.
But anyways, great work and great idea! Kudos, well-done!