There is certainly room for IDEs to improve. But there's no way we can ever have the level of automation and abstraction that photographers, video editors and other creatives enjoy.
Take the photo clone tool mentioned. Why does that work? Because human vision is lossy and forgiving - the difference between 95% correct and 100% correct is around 5%, depending on how closely the person is looking at it.
Programming on the other hand... code is either 0% correct or 100% correct. Anything less than 100% correct is broken.
Media is also far less complex. How do I know if my video is correct? I simply play it. If there any errors, they are all possible to detect in the minutes it takes to play the video.
There will never be an equivalent for code. You can't visualize an arbitrary algorithm over all its inputs - that's undecidable. It's hard enough to see what a function in three variables does, let alone quicksort.
The best we can do is have editors for specific tasks. GUI editors, markup editors, etc. For everything else, code is the only option. We can patch the awkwardness with better languages and better IDEs, but it will never, ever, go away.
"Programming on the other hand... code is either 0% correct or 100% correct. Anything less than 100% correct is broken."
I don't think that's true.
Anything less than 100% correct is incorrect, it isn't necessarily broken.
How do I know my program is not broken? I run it. If it produces the answers I want, I can generally assume it is not broken (with a dose of skepticism).
If it is actually incorrect, it could still compile, run and/or produce useful answers (it would depend on the input). Almost all of my code has subtle bugs that I have had to iron out at one point or another.
I know, I know;
What about space shuttle control systems or critical airplane frobnobdicators? Fine. Those aren't lossy and forgiving, but systems like that are few and far between. There is a large subset of programs that are also lossy and forgiving. (Which I assume the majority of us work on).
Were it really a binary reality, we would have much less software.
I was talking in terms of transformations, such as the clone tool. Clone takes a set of pixels, applies some math, and gives you a new set of pixels that look a bit like what you want. After you've applied clone, you're done (although you might need to apply some finishing touches).
There is just no room for such tools in programming. You can't have a tool that applies some transformation to the code but leaves half of it in a broken state. That would be worse than useless. No, a tool has to be 100% correct or introduce errors that are caught by the compiler, otherwise it'll take longer to find and fix its mistakes than it would to apply the transformation manually. And it turns out that making such perfect tools is incredibly hard.
How do I know my program is not broken? I run it. If it produces the answers I want, I can generally assume it is not broken (with a dose of skepticism).
But notice how much effort is put into making sure code is correct. We have unit testing, integration testing, fuzz testing, static analysis, formal verification, ... Games studios pay scores of people to sit around playing games all day. This is not seen in any other creative art.
Uh? I don't know about vim, but emacs doesn't even get syntax highlight right for most languages.
It's a great text editor, and I appreciate it as such, but it understands way too little of programming languages other than Lisp to deserve being called an IDE.
Emacs doesn't really come with out-of-the-box great support for most languages. Even for one language there are usually several alternative modes available, from which you have to choose. Even for Lisp you probably want to consider SLIME instead of the default.
Unfortunately not every popular language has a truly great mode available. And the whole mode-ecosystem feel like a pita (especially for newcomers), but at least you have a choice, which you don't have with most other editors/IDE-s.
But your talk about syntax highlighting is just unfair. I for example use a JavaScript mode written by Steve Yegge, which actually contains a full-blown JavaScript parser which pretty much ensures that under normal circumstances I don't make syntax errors. Oh... and it also has syntax highlighting :)
I was talking in terms of transformations, such as the clone tool. Clone takes a set of pixels, applies some math, and gives you a new set of pixels that look a bit like what you want. After you've applied clone, you're done (although you might need to apply some finishing touches).
The pre-packaged refactorings in the Smalltalk Refactoring Browser are like this. There are checks to see if the refactoring is guaranteed to be correct, and if the refactoring runs, you can be very sure it is. The refactoring commands are even undo/redo-able.
"But notice how much effort is put into making sure code is correct. We have unit testing, integration testing, fuzz testing, static analysis, formal verification, ... Games studios pay scores of people to sit around playing games all day. This is not seen in any other creative art."
You would have all of these things with or without a 'clone' tool that produces approximately correct code. As long as it produces code that is in some way human readable, can be debugged using standard (above) means, and doesn't make too many mistakes, it would be fine (no worse than me writing code, at least).
I think that your assumption that it would make a lot of mistakes and that they would be tedious and difficult to debug is just that, an assumption. It might be as simple as the programmer simply 'touching things up'. We haven't seen the tool, it doesn't exist, so arguing about what it would do is a moot point.
> Anything less than 100% correct is incorrect, it isn't necessarily broken.
> run and/or produce useful answers (it would depend on the input)
The fact companies can and do sell broken products that generate wrong answers, lock-up or crash from time to time has no relation to the fact the products are broken.
An old joke said that IBM stood for "Inferior, but marketable"
You can have code that is not 100% correct, it still runs and generates reasonable answers, without crashing, and is very useful. Even code that has been worked on for years still has subtle bugs lurking in it.
Take the Linux kernel. It has bugs in it. Somewhere, there is a bug! I'm 100% sure of this. Same with Windows, OSX.
A non-trivial piece of software with a bug is a tautology. We don't say that these products are broken, because then all software products would necessarily be broken.
Sure, in software development we are doing our damnedest to approximate correctness, but we never get there.
A tool that writes approximately correct code would be very useful.
How would it be useful to me, if I spent my time scrutinizing and debugging the code output by the tool?
I'm sorry, but this kind of inconsiderate, care-free attitude about bugs is an obstacle to me as a developer, to the brink of being downright offensive. Assuming of course we're writing code which is more complex than Hello World.
All I want, as a programmer, is a tool which aids me in reading, writing and manipulating code, without getting in the way. A clone feature is completely useless to me; a proper IDE has (or would have, actually) a library of code that I can choose from for my purposes, or perhaps would let me choose from my own snippets I've written in the past. Simply letting me drag a function to the point(s) in the code I'd like it to be called and then placing the necessary #include directives and/or the necessary compiler/linker options, and perhaps taking care of some parameters based on default values, a conceptually simple task, seems like science fiction to vim or emacs.
It's utterly stupid. The article is right on. We're wasting our time on syntax errors.
"How would it be useful to me, if I spent my time scrutinizing and debugging the code output by the tool?"
How would it be useful to me, if I spent my time scrutinizing and debugging code put out by myself?
I am going to do the scrutinizing and debugging either way!
How is my attitude about bugs carefree? They are everywhere. They are inevitable.
That isn't carefree, those are the facts of life. You screw up, I screw up, Tom screws up.
(I consider myself more of the paranoid camp wrt bugs, personally).
I agree that at a minimum an IDE has to do the things you listed.
If I had a tool that generated code pretty much as good as mine, that I could debug and ship, where would the loss be? The tool doesn't exist, and probably won't exist, but if it did, it would be awesome.
"How would it be useful to me, if I spent my time scrutinizing and debugging the code output by the tool?"
How would it be useful to me, if I spent my time scrutinizing and debugging code put out by myself?
------
Small point: I'd much rather debug errors I introduced than ones a program did. If I made the errors, they will have a familiarity about them - "oh, I always misspell that," or "I wasn't paying attention there." I can't predict how a program might get it wrong.
I'd much rather debug errors I didn't write myself, they are much easier to find. I find that code I wrote (often) scans the way I intended to write it, not the way I wrote it.
<sarcasm>
In other words your code validates past computation and adds error correction code to all files you have to the HDD? After all if you don't do that then it will often produce buggy output and crash. (http://www.pcguide.com/ref/hdd/perf/qual/specRates-c.html)
</sarcasm>
At best software running on commodity hardware works most of the time and acceptably broken really is good enough. Granted we want to minimize failures, but there is a wide range between good enough to be useful and perfect.
Yeah, that's why we don't have anyone fixing bugs encountered in software we use.
Please.
PS. Who cares about perfect? You can't have perfect, and that wasn't my point. Please reread.
PPS. I'm happy to see you're invalidating all the subject of my current work (carrier-grade systems) based on what's good enough on commodity systems. Extrapolating is bad.
PPPS. Sarcasm tags are so 17th century Protestant.
That mindset often results in over engineered crap.
I once wrote a program to parse a source code in ways that is provably limited. However, based on way the code was written it worked often enough to be extremely useful. Just because an approach can never cover all cases does not mean must be ignored.
Sniper bullets have tighter tolerances than the average bullet but this increase costs and becomes useless at high rates of fire. Suggesting all systems need to be as accurate as possible is ignorant.
PS: I have written and used plenty of code sitting on DoD computers and believe me it’s not all golden. Walk around the Pentagon and guess what Windows is sitting on the vast majority of peoples desks.
Again, you're extrapolating just to prove me wrong. I admit it may seem 'extreme' to some, but advanced tools aren't a threat to us. However, our current culture as programmers shows that we don't even trust other people's source code, let alone tools, to do our job. In this context of today, a tool that "writes code" (imperfect, of course) is not to be trusted any more than me, you, or our colleagues.
But do not downplay the importance of not producing, fixing, and avoiding bugs. It's just wrong.
There are basically two ways to write software you can either completely understand the problem and create a completely understood solution, or you can approximate a reasonable solution and then patch. Now when it’s possible the first approach is far better, unfortunately some of the worst systems where created when people tried to use the first approach when it was simply too complex for any one person to understand.
If you have ever copied a function changed the name and modified the code you are on the second path. Now plenty of people have gone down that path and know it’s likely to produce bugs, but as long as you have reasonable mitigation strategies it can still be a good idea. Building a tool that does similar things with larger sections of code would be dangerous, but with a little care it could still be useful.
In the early days of Fortran compilers they tended to be buggy so people would often money patch the output. http://en.wikipedia.org/wiki/Monkey_patch While error prone this was significantly faster than hand coding in assembler from the start. This died out with the invention of better compilers, but plenty of good software was written before then.
PS: I wish most software could be written using the first approach. I just don't think that's possible.
I don't think I really agree with any of that. The binary aspect of software workingness doesn't really matter... Code is just bits of information that gets massaged around, there's no reason why the tools to do so can't improve.
Sure the specific example of visualizing an arbitrary algorithm over all inputs might be a little unrealistic, but the point of the post wasn't that we need that specific thing.
I'm not denying it's a complex problem (grokking what the code does is hard already, nevermind grokking what it should do), but complex problems aren't fundamentally impossible, they're just... complex!
Sure the specific example of visualizing an arbitrary algorithm over all inputs might be a little unrealistic, but the point of the post wasn't that we need that specific thing.
On the contrary, this is exactly what we need. Easy visualization is the reason why word processors, photo editors, video editors, music notation and audio software, DTP packages, and 3D programs are so much more powerful than IDEs. WYSIWYG gives the user a view of the entire state of the document they're working on. Users can see what's wrong or what needs doing, then fix it.
It's certainly possible to create this in an IDE in a small number of specific cases, but it's totally impossible in general (and I mean this in the truest sense of the word "impossible").
I agree with you and the author of the post that we should definitely work on getting IDEs smarter, and there's a lot of scope for improving them. It's just that computer science places a hard limit on how far we can go.
I entirely disagree that visualization is a/the missing panacea.
Code is not a visual thing nor does it even represent a visual thing -- making it visual accomplishes nothing (except for making it visual).
Pictures, formatted document text, 3d models and videos are visual things. Music notes are a specific type of visual language, audio is time-linear-spatial (or something less gibberishy like that) and can be represented visually. These things are all extremely different from software.
Surely it will be really hard to get better ways of visualizing code, it's an abstract thing and so it's hard to represent.
But what we are really missing is execution visualization. Applications aren't code, applications are execution systems and it's a shame to test execution/debug like we do today.
As someone has said, it's almost impossible to test all inputs for an algorithm, it's impossible to prove the correctness of an application without testing it.
Without testing we're nothing, most of us can surely say in which conditions our applications will break, but how sure are we to say in which conditions will it work? Much harder.
I would love to work in IDE improvement as I believe there is really a lot of things to improve.
Slow, inefficient code. Poorly written, indecipherable, spaghetti, unmaintainable code. Undocumented code. Untested code. All can be (in theory) not broken, but are far from 100%
"Programming on the other hand... code is either 0% correct or 100% correct. Anything less than 100% correct is broken."
I don't think that's even remotely close to accurate.
Spend a day looking at the Daily WTF, and consider that the Daily WTF is a LOT more representative of the IT world than HN is. There are raftloads of garbage code out there, and the people writing it can get away with it because it kind of works, even if most coders write 2000 lines to do what a competent coder would have done with 200.
In visual effects, the viewer is far LESS forgiving. Why do computer-generated water effects, even with multi-million dollar budgets, still look CG? It's because we are so familiar with what water looks like that we subconsciously pick up on miniscule flaws that are exceedingly difficult to groom out, because the computer-generated water requires so much computational power.
The IT industry as a whole is stark proof that code is far too forgiving. Look at the host of rants about corporate IT -- the vast majority of IT, and the reason that "enterprise" carries the connotations of certain 4-letter words. And look at films that are heavy with visual effects, and see how easily you can find flaws in them, and how much effort film makers put into creating those animations.
The difference? You can put broken code into production, and get away with it easily. Outside of communities like this, let's be honest -- hardly anyone can tell that the code is utter garbage, proven by the fact that most of it IS utter garbage.
Then look at how easily you can detect flaws in almost any visual art medium -- you might be surprised at how much effort goes into crafting a fine art photograph, given how easy it is to allow miniscule flaws to break the connection with the viewer.
This would be fine if it were obvious to all of us that GUIs have advanced as far as they can, and the remainder of the problems we see in them were inherent to the complexity and nondeterministic nature of the systems they manage.
But it's not like that. My IDE occasionally gets confused and forgets how to rename a file. Eclipse takes 3 minutes to refresh a source-file tree. People still use vim and emacs for things! There could be no clearer sign to me that IDEs are rubbish pieces of software than the fact that people keep falling back on tools that have barely evolved in 30 years.
You are kind of making the assumption that there should be something better than Vim and Emacs because they where created decades ago and are not pretty "gui driven" applications. Programming is simply writing and editing text. Vim and Emacs are editors and they fulfill their jobs marvelously, and that's OK. We don't need no stinking IDEs!
I agree with most of what you are saying, but never, ever is a long time. If I can be optimistic for a moment, I think 100 years or more from now, successive languages and IDEs won't simply be patches to awkwardness, they will probably help with many real programming problems significantly. Ask yourself how many times have you re-invented something over and over again, or the fragmentation of documentation and usage overhead of frameworks. These are just some. Computer science and software engineering are young disciplines.
Comparing different IDEs is often a matter of style and personal choice right now, but if there is ever a big milestone, many will know it and many will adopt it, but until that point I don't think being complacent is the answer. Who knows what black swans lie in the future.
> Media is also far less complex. How do I know if my video is correct? I simply play it. If there any errors, they are all possible to detect in the minutes it takes to play the video.
There's actually a level of complexity to video well beyond what you imply. For example, making sure the colours appear the same when you watch something on your laptop screen and when you watch it in the cinema requires a lot of care and attention - and no small amount of science. Also not all video edits are realtime - some things require a lot of processing, which can take just as much time as compiling a decent sized app.
Yeah, anyone who thinks that media is less complex or less demanding than programming should spend a day doing some visual effects work for a live action movie (try creating a virtual character sometime) or talk to the guy I'm working with to do my photo finishing.
The reality is that it's far, far more demanding than programming, especially with modern languages.
With proper up-front design effort, programming is easy. The catch is that the up-front design requires engineering skills and is actually challenging, and most of today's programmers skip that and dive right into programming... leading to the vast mountains of crap we have today. (Look at amazon's codebase sometime. It's such an atrocity that it drives people out of the industry.)
This is a great point. For me it isn't poetic at all, but a deductible consequence.
If our job is creating abstractions it must be really hard to create tools which abstract properly our job. Where abstract properly would be separating what is really our job, the abstractions we are creating, and what are the trivial abstractions, the ones we assume as stable. This is a hard path to reach any usefull conclusions or insights to develop new programming tool features. The separation from what are the abstractions we're creating and the abstractions we're building on top off is always changing and it's in the same place (in the code).
For reaching usefull conclusions I would rather think in the different tasks developers need to pursue. Who wants to build a RescueTimeesque tool for developers?
Lisp, with his code is data super-feature, and his REPL super-environment has a lot to say in this two issues.
- The file system; you don't need one when you're building an HTML prototype. Instead you have sheets, where you can organise multiple "files" (URIs, basically) based around how you want to organise, not how some arbitrary hierarchy of nested folders wants you to.
- Code completion; intellisense and other completion usually pops up an alphabetic list of everything available in the namespace. In the case of CSS, typing "a" presents you with autocomplete starting with "azimuth". When do you ever need azimuth? So instead, we built "common sense code completion", where the likely most-used properties are surfaced at the top of the list. Now, writing HTML and CSS is mostly a case of pressing tab a lot, and up or down when you don't want the auto-suggested default.
- Browser-based; why even write code in a desktop IDE, especially HTML/CSS? Doing in the browser has so many advantages, such as web browsers' awesome tab features or, since it's a web app, everything you write being instantly available online.
- Collaboration; as a web app, no more checking in and updating. We handle version management and you just work on your code with your team (long term: Google-style realtime collaboration?)
There's a lot to improve about today's IDEs. We got started on one specific for simple HTML development. Someone else should solve this problem for programming languages.
Visual Studio with Resharper does half of the things mentioned in the article.
Some of .Net, LINQ for example, was built with the IDE in mind hence LINQ's 'backward' seeming query syntax (you say what you're querying over up front, so the IDE can provide typed autocomplete/information).
Even with VS and Resharper, there's still so much that could be done. It's a massive problem though, I think.
Agreed. With ReSharper, I don't recall a time where syntax was incorrect at compile time. It immediately indicates any syntax errors at coding time.
However, the inline method/function definition that the OP mentions is something I have also wanted. The two closest features are:
1) Visual Studio has a 'Code Definition Window' (http://msdn.microsoft.com/en-us/library/f5yx24a6%28v=VS.100%...) that dynamically displays the contents of a method in a separate window pane. Although it is not inline, at least you can visually see the code without navigating away from the current source file.
2) In ReSharper, you can sort of fake it by calling the command 'Inline Method' and unchecking 'Inline all usages' and then Undo when done inspecting the code. However, that is a hack with its share of problems. I'd prefer a feature to toggle the inline method content in a preview mode without actually modifying any existing code.
> However, the inline method/function definition that the OP mentions is something I have also wanted.
Eclipse actually does this. If you hover your mouse over a function name, it will pop up a tooltip showing you the code for the function. I find it extremely handy - it's one of the things I think I'm really going to miss, having just given up on Eclipse because of its frustratingly bad performance.
It seems like the author forgets that the medium of code IS text. Not graphics, or sound, or a 3D model... but code is text, and for now text is the most concise representation of code.
But actually, a textual representation means that you need to rely less on fancy GUI's to write code. There are also added benefits like ease of search. If you're using a nice editor like Emacs or Vim, the medium is very easy to manipulate.
It seems like the author forgets that the medium of code IS text.
In fact, it isn't. Code is better defined as a tree. The "impedance mismatch" of writting a tree using sequential text strings is what makes fragile the tools that IDEs feature.
That code is seen as text is just a historical accident. I believe most serious programmers have been beaten by some incomplete implementation of graphical interfaces and accepted to fall back to good old plain text.
But we are in 2010. This acceptance of this state of matters is very unfortunate. Specially sad is when the author says that IDEs are no more than text editors with some extended capabilities. They shouldn't! They should load the code in all its glorious shape... that by no means is a chunk of text.
> Or are you saying we should program in visio-style diagrams?
Yes. It should be possible to develop an interface that is actually faster and cleaner to program in than text. I'd love to see what touch interfaces can do in this regard.
It's never been done, granted. But it should be possible. I hope developers aren't dissuaded from attempting graphical programming interfaces just because everything that has existed up to now has sucked.
Syntax highlighting is pretty much it. Adding much more would blow out our ability to handle complexity and just be too busy on the screen for anything to make sense.
Go look at a language parser sometime; we've actually put a lot of work in any production-ready language into that "low tech" text interface. The tree approach is useful for macro writing, and if you don't know Lisp maybe you should go check one out, but I'm very unconvinced there's a better way to actually interact with the code even in theory. It may be visually unimpressive but it's actually incredibly, incredibly sophisticated under the hood. If it's good enough for human speech, it's more than good enough for programming.
Yes, I love lisp - I'm a huge Clojure buff. And that's exactly what makes me think a graphical programming interface could be done well. As languages get more lisp-like, it seems as if they would become more conducive to a graphical programming environment.
I imagine looking at something that looks a lot like lisp - it shows the high-level structure of the code as a tree. But it could allows me to change the level of abstraction I'm viewing on the fly. Also, it could reveal (or hide, at my preference) lots of metadata about the code... is it pure, is it stateful, is it parallel, etc.
In essence, it allows you to look at your code as an n-dimensional entity instead of a 2 dimensional file.
Basically, I'd like to see what happens when you decouple the code from its presentation. Then, you can choose alternate presentations (including plain text!) depending on the task at hand.
That's just an imperative program using "blocks" instead of "lines."
An innovative, usable visual programming language will introduce a new programming paradigm (probably based off of functional languages) that provides deep integration with the GUI and plays to its strengths, not just do a linear line->block transliteration into a graphical environment.
Although, Scratch is great for teaching kids about programming.
Look at XML (its simpler than programming languages so easy to discuss). If your sole editing tool is a text editor it will suck - things like unclosed tags are possible.
If your editing consists of something like elementtree, that actually works at the higher level of the language, pushing and popping elements onto the ends of xpaths, you might actually not mind editing XML at all.
No one's discovered a better method of representation.
Hmmmm... that's only half true. The text layout in a real program is already a tree-like representation: using vertical and horizontal space (blank lines, indentation) and providing syntax highlightling, we "paint" text to resemble the real structure: the one the compiler understand.
But syntax highlighters, "intellisense" and the rest of IDE tricks are just that: a patch over the text handling.
A true IDE would read code directly into a in-memory tree structure, resolving external references, making a first processing of the code on the moment the code is loaded.
A true IDE would eliminate the burden of writing annoying control characters (curly braces, semicolons, quotes) to placate the compiler, escaping delimiters, correcting indentation, etc. The editor could use some quick key combinations to navigate the code.
This is not really related with a Visio-like interface. It would look more like add-ons such as Resharper that "understand" the code to a point and make it easier to write code. Visually it would present the code already indented and colorized. It could be "skinned".
Unlike add-ons, it would be fully aware of the code meaning, so it wouldn't need scanning the code to support editing. The code would be fully loaded and cross-references checked.
BTW, there have been some proof-of-concept of "syntax-driven editors" or something like that. I haven't even seen them, so not sure to what point it's what I have in mind.
Even though code is text, you would think someone would have tried to build a different view given the popularity of MVC.
Inline expansion would be nice. So would code paths. So would inline images & html rendering for documentation. So would the relevant rendered html for the issue relating to the code you are looking at. So would other annotations. And what about visually hiding code when working in a language like java?
I tried to get this going in eclipse once, and it really wasn't going to happen easily so I gave up. I've always hoped someone would build something similar as I feel it would really help productivity having everything relevant right there with the code.
Note: The project http://code.google.com/p/lambda4jdt/ is the closest thing I have found to what I am suggesting which I think shows the power. It just goes to show that it could work.
Indeed, MS are trying to encourage this with the new WPF-based editor in VS2010. The developer's examples are all about graphical overlays replacing certain types of code. It will be interesting to see if ISVs take this forward.
Let's say you had an eclipse plugin that let you write in lisp, but it actually converted the code to java underneath. You could click a button to switch between views. Are you saying that would be a bad thing? If a tool allows you to be more productive, I use it. Note: I don't use lambda4jdt at all, I just like the idea of creating problem specific views for code and I don't think that belongs at the language level.
You're right, my comment was directed at your example really; the code before and after use the same representation, text. In that case it seems the original was simply a poor textual representation of the logic. I'm actually a firm believer in being able to visualise code. After all coding really gets done in my head and when it's in there it's in 3D images/videos. I long for the day when I can work in an immersive environment where I can walk around the design suspended in the air. That day seems a long way off at the moment though :)
Given that IDEs are the primary tools of programmers, and unlike Photoshop users our primary skill is to build software itself, why are IDEs not the most advanced pieces of software that exist? It seems logical. They have the most advanced users; they don't need to worry about scaring the kids at home. They should be constantly adopting the latest and greatest in performance and UX.
Instead we have vim and emacs which, beloved as they are, still have as their primary interface a grid of characters -- not just the code itself, but the user interface. It's like nothing has changed in terms of the graphical output capabilities of computers since the late 1970s. (And yes, there are WIMP versions, which add drop-down menus. But the primary interface mechanism is still keyboard shortcuts.)
Then there is the beast called Eclipse. It certainly seems like it should be able to do everything in the world, certainly the download takes long enough and there are enough configuration options. Yet it is buggy, its plugin architecture is a conflicting mess, and its performance is so abysmal that frustrated coders return to vim and emacs so they can get something done.
Version control -- a critical function for all modern programmers -- is still largely handled on the command line. Maybe, if you're lucky, you get a side-by-side comparison in Eclipse (but not for git, yet!), but as far as tagging and branching goes, graphical representations are out. WTF?
I'm so glad to see this article, because I have spent so long in the company of programmers who hear me ranting about IDEs and just tell me to "use vim", as if that were the problem.
Version control - the most text-oriented part of the whole deal. Files of ASCII, tracked by LINE???
Instead track definitions, APIs, formal argument structure, constraints! Make all that and more part of the 'code' and track its changes and meta-changes. And check it against the product and test plan too!
I imagine a schema capturing the semantics of a program with an audit trail, checkpoints of consistent state, reports that flag contradictions and incomplete subtrees etc.
Instead I get "automatic merge tools" that shuffle my changes into somebody else's like a dealer at Vegas.
Version control on a semantic level would require to understand the semantics of source code. Thus, your ideal vcs tool must have a parser for every programming language, every XML schema and other non-binary data format.
Exactly. A parser for each programming language is the cheapest, simplest part.
At root, programming languages all come down to a semantic graph. Manage that! At the same time, you would be able to render the 'code' as any language you like (within reason).
Or it supports X language provided by Y plugin and per-line versioning elsewhere. Does a python programmer care if his magical VCS tool doesn't understand the semantics of !python code? I think the parent's suggestion is quite novel!
> Then there is the beast called Eclipse. It certainly seems like it should be able to do everything in the world... Yet it is buggy, its plugin architecture is a conflicting mess, and its performance is so abysmal...
I have used Eclipse for many years with multiple languages, and none of this is quite true. There is a bit of a learning curve, where you have to learn not to do some things that break it. :) But that just takes a few hours, and a bit of patience - after that you have many months of happy coding, where it just works...
I used Eclipse daily for about two years, along with an internal C code plugin. I'm not sure which I can attribute to Eclipse and which I can attribute to the plugin, but my Eclipse workspaces were frequently getting randomly corrupted, global search-and-replace was not trustworthy, and for a while it was running so unstable overall that there were times I could launch Eclipse, load my workspace, and within a few minutes of just sitting there it would crash. This was running on Windows XP.
So I tried Eclipse at home on my Mac. I saw that it came with a utility for auto-packaging Java programs into OS X bundles. I made a simple Java Swing "Hello World" program and tried the bundle utility. It didn't work. I tried a few more times, and it never worked, but in fact got progressively worse in its output. Uninstall.
Some months later I thought I'd try Eclipse on the Mac again; surely it had improved. So I downloaded it again, created a new project, clicked through to add a new source code file and I got an error, that it could not create a new file.
I'm guessing that my experiences must be abnormal, since Eclipse is so hugely popular, but I now avoid it whenever I possibly can.
Yes, your experiences are extremely abnormal. I've used Eclipse extensively (I think since 2002 or so) and apart from the very early days I have never had a corrupted workspace, I have never ever had a problem with inconsistent search results and I can't remember an occurrence of it crashing (unless you count an out of memory, which is just from bad config).
Those are pretty strange. I am using it on a Mac right now, with no serious problems.
I'm not sure about the bundle creator, I never used that, but not being able to create a new file seems like some random little issue that's most likely fixable.
Since we're talking anecdotes, I've never ever had any of these problems. I've used SVN/Git plugins, Android's plugins, etc on every operating system that Eclipse supports and I've never ever had a problem.
I used eclipse to code java every day for 5 years. I used it on windows, linux and OS X. During that time I upgraded computers a few times. Each time I upgraded I was optimistic that my shiny new machine would make eclipse run at a tolerable speed. Having a language that was succinct enough to not need a beast like eclipse was one of the main reasons I originally switched to ruby. Using something simple like textmate sometimes means I have to go the long way round when I want to perform a refactor but at least I know when it's going to be slow. With eclipse you can do something as simple as open a file and have to endure 30 seconds of freeze. When you need to get something done fast that's painful not to say stressful!
Either you have a v slow machine or you had some major configuration issues with Eclipse or other services running on that machine. Even on a 1.4GHz Core 2 machine on a large java project, source files open < 1 sec.
So you seem to be Eclipse-proficient. Please, tell me: How do i upgrade Eclipse (3.5 to 3.6) without reinstalling the whole thing and all plugins manually one by one? Why are plugins downloaded with 20KB/s, where apt-get gets 1MB/s?
Is there some serious reason you need to upgrade? If so, then I think you can just unzip a fresh install and copy over the plugins you want (as .jar files) into the new directory structure. Apparently there are more ways to do this, though I didn't try: http://www.venukb.com/2006/08/20/install-eclipse-plugins-the...
> Given that IDEs are the primary tools of programmers, and unlike Photoshop users our primary skill is to build software itself, why are IDEs not the most advanced pieces of software that exist?
I guess it depends what you call advanced. There's some pretty impressive work that goes into the static and dynamic analysis, refactoring, profiling etc. tools that you see in modern IDEs. Is that more or less advanced than software that can identify faces in an image? Who's to say?
BTW I think you're being a bit unfair to Eclipse. I agree about its performance, but picking on the download size seems a bit disingenuous: looking at the download page right now the largest package is less than 250 Mb, no more than a few minutes even on a slow broadband connection. Both svn and cvs are well supported within the IDE too. I've just decided to switch away from Eclipse primarily because of its performance, but it deserves justified criticism at least. :-)
By comparison, take a look at Blender3D. It's a 22MB download, though you need a Python interpreter, which is also not all that large.
Eclipse is impressively bloated, but that's about the most positive thing I can say about it nowadays, especially after being stuck using it at amazon and disney. (Both of which had some of the poorest quality legacy code I've ever seen.)
Adobe has probably made billions of dollars on Photoshop, and has thousands of people working on improving interaction with a regular grid of color values. This is a high profit, low complexity situation. Most developers are not willing to spend $700 on an editor, and the gamut of programming language source code syntaxes is too broad for a single editor to work uniformly across them. But really, comparing anything to Photoshop is silly.
Some music sequencing programs will in fact allow you to create measures with the wrong number of beats, and have annoying, unproductive interfaces. Premiere is actually really annoying to work with, which sent a lot of people over to Final Cut. Even extremely similar programs like Adobe Illustrator are fraught with miserable design and crappy storage formats.
The OP has valid points, but these apply to most software, and not just IDE's.
I'd spend $700 on a truly good IDE, and so should any programmer capable of doing simple arithmetic. Calculate how much more productive you can be using the best tools available, then figure out how many hours at your bill rate it will take to make back that investment.
For something like IntelliJ ($$$, but awesome) versus Eclipse (free, but crap), most people will find that they break even in less than a week. Assuming you expect your programming career to be longer than that, it's not a tough decision to make.
A few programmers will pay a lot of money, but most programmers seem to be uninterested in paying more than $70. Compared this to photographers, who will almost universally pay $700. The market is much larger.
That's only like 35 copies of Photoshop. Not to downplay the money you spent, but the market for Visual Studio is smaller than Photoshop. Also, the customer of Visual Studio is team leads and management, not actual users. This might explain why more expensive VS packages emphasize collaboration and team features.
VS.NET is something of a special case, since Microsoft would like nothing more than to give it away for free. They don't, because if they did it would make toolmakers like ReSharper and CodeSmith seem unjustifiably expensive.
If you're a small shop, you can get in thru BizSpark and get a pack of MSDN licenses big enough for your whole team for something like $400, all in.
So sure, you can pay $2500/copy retail if you're big enough to do so. Or if you're small, you can pay peanuts. Say what you want about Microsoft, but they go out of their way to take care of their developers.
How come I can still have syntax errors? How come it is
ever possible to have a syntax error in a program? Shouldn’t the IDE at least be helping me to have a valid
program all the time? Finale doesn’t let you write a
measure with the wrong number of beats and then complain
when you push play. It just fixes it for you — “oh look,
you need a rest there.”
====/QUOTE
Oh Luke Palmer, no.
NO. The absolute last thing I want from my editor is for it to be modifying the code without my explicit request. Then if something breaks, I need to worry about whether I broke it or whether the editor broke it. (And yes, yes, I know, the compiler "modifies my code", blah blah.)
I think it is a case of apples and oranges here. Yes we need to make progress on IDEs, and of course we can borrow inspiration from many sources. I've used every application the author has mentioned: Photoshop, Premiere, Maya, AutoCAD, ProTools, Finale, Reason, PageMaker, and the reasons the tools succeed have more to do with the necessities dictated by the media you are editing rather than some kind of profound innovation.
If I remember correctly, in one of the SICP video lecture, Hal Abelson says that s-exprs are equivalent to the the syntax tree of the program and, given a graphical interface, we would be manipulating this tree directly. AppInventor for Android is the most similar thing we have today, and not surprisingly it's based on Kawa scheme. It would be a very powerful mode of code editing, if it wasn't centered around click and drag operations (and limited to android). I think that a modal editor ala VIM, paired with a graphical code-block environment and fuzzy text autocompletion, would be a nearly perfect IDE.
I'm perfectly happy with the tools I use when programming.
The tools I use to manipulate images and photos on the other hand... they're big and clunky, force me to use constipated shortcuts, are tedious to install, cannot be run remotely (well, effectively), they crash and aren't that easy for me to extend.
I'm, sort of, working on the opposite problem: I'm tossing a few hours towards a way to manipulate images without the "IDE".
How about things like ImageMagick?[1]
Command line or from a programming language. No IDE at all.
I've used it in the past, and it's useful for batch operations.
So now I'm thinking the main problem is that it's hard to compete with open source. There are just so many good-enough open source programming tools out there. Commercial tools not only have to better than the open source stuff, they have to be way better. This is because of all the costs involved with adopting a new IDE: Not just the price of the software, but the time spent learning a new environment and getting it configured right are significant time investments.
So open source eats away at the margins of commercial tools, which leads to less people working on commercial tools, which drives more people to the open source incumbents, etc.
I believe pg says open source is bad at design. That seems right to me. Open source is good at copying existing tools, but not so good at innovating. So we end up at a local maxima where all programming tools are decent, but not great.
The author really is understating the power of things we already have, Visual Studio and Emacs in particular (and I suppose he would clump Vim in as well). While they aren't the holy grail of syntax checkers, they are still extremely well tuned tools for creating software.
Agreed. I'll throw in my vote for Emacs; you can get very far with autocompletion (M-/) and tags. Autocompletion will complete the word you are typing with other words from all your open buffers, and it works 95% of the time. Tags allow you to jump to definitions easily.
The way Emacs implements these are so dumb but in reality they work great, and it's all really fast. The problem with large IDEs is they can be really slow if they are having to constantly analyze large codebases.
Oh boy, and who could forget DWIM. I'm sure if the concept of refactoring existed back then (it didn't; apparently L Peter Deutsch would just throw out your code and start again if he didn't like it) there would be some really cool tools for that too.
For all the hating, automating programming is an obvious thing to do (the idea is both trivial and quite profound: http://news.ycombinator.com/item?id=645338), and there are some really cool ideas there (reversible debugging being one).
Some creative apps are dataflow systems. Dataflow maps very well to GUIs and intelligent tools. For the parts which aren't dataflow, you end up with the same manual labour that you have in code editing.
Video editing, for instance, is quite similar to code editing. Juggling large libraries of assets, trying to make things fit together on the level of details and taste while maintaining an intelligible and manageable whole and keeping to a deadline. You have tools to help you keep track of things, but you're still mostly on your own.
My personal IDE wish list below (some simple, some not so, and necessarily incomplete) -
1. Copy-paste abstraction - When you copy a snippet from one part to another, automatically lift it out as a function first. If it can't be abstracted for some reason, alert!
2. Path finder - I have A,B,C and I want to get X,Y,Z. Find me a path through the libraries please. We created Prolog. Can we use it for this? Even guesses are useful. Hoogle is a start (thanks Curry and Howard!). The ABCXYZ can be abstract descriptions like type, or example data.
3. Debugger time travel - move back and forth in time freely, allowing code modification. Me got tons of RAM+disk for history and branching. .. and why can't something get unwritten from a file?
4. Breaking news - when I modify a function that could break something else, tell me now.
5. Link function arguments to in-source example data for testing and exploration. I love to start with data first before writing anything.
6. Embed images and live code snippets as documentation. Ever drew ascii diagrams in source code? Mathematica notebooks have solved this and more already.
7. Function grapher - select an expression and show as graph. At least what "Grapher" on MacOSX provides. The IDE should know basic mathematics - like mathematica.
8. (extension of 7) Design at least simple functions graphically.
The adobe tools mentioned are excellent for what they do, but they are never without interface quirks that can range from mildly irritating to bizarre -- I have consistently found the Edit Styles box in InDesign to work counterintuitively in small ways, like forgetting that I checked 'preview' or overwriting my edits if a dropdown has been incorrectly fiddled with.
Adobe stuff also completely lacks decent version control. "Version Cue" is pretty clunky and has no conception of diff-ing. If I make edits to the story in an InDesign file and replace 3 FPO images with real ones, why can't I have a repository I can browse, in which I can see an awesome and self-explanatory visualization of these changes? Maybe you need InCopy servers for something like that but I don't think that's a feature they have.
Contrastingly, with the latest XCode preview, versioning was almost seamless and very nice to work with. After installing the XCode preview, I was able to check out a project from GitHub, run it, edit it, and commit the changes, with a lovely two-pane view of the diff in question when committing, showing additions and deletions. I didn't have to think about it at all -- it was only "almost" seamless as I had to manually do 'git push', which I'm sure they'll fix that in the final release.
You can do scripting with JavaScript inside the Adobe apps, but it's slow and the API takes some considerable getting-used-to. Some things most programmers would consider scriptable are not: I'd love to be able to bust out a regex or some conditional logic when building a Photoshop automation, for example; the current interface for that feels condescending and ridiculous if you're coming from TextMate or emacs.
He's focusing on tool support (for manipulating representations of code), but languages (representations) could do with improvement as well.
Or perhaps you could think about both together. How would you coevolve a language with tools? If you could make assumptions about the tools people have, how would that change the language?
Oddly, I have been thinking exactly the opposite - what aspects of IDE's are missing from graphics programs.
Among the features of IDE's that programs like AutoCAD and Photoshop lack are continuous error checking and validation. They provide little feedback regarding proper document organization or standards and specifications compliance as part of a typical work flow.
In addition, some IDE's offer direct visual feedback regarding user interface design along the same lines (though not necessarily to the same degree) as graphics software.
edit: I'm not saying that these features are impossible to implement, rather that implementing them is not feasible for the typical user(victim).
Any other VS.NET/ReSharper folks get halfway through his list of things that IDEs don't do, quietly noting that our IDE does in fact do them, then quit reading?
That's one of the big benefits of a compiled language like C#. The IDE can be continuously background-compiling, making things like Refactor/Rename work like the author wishes it would instead of replacing strings. Same with Find Usages, Find Declaration, and Automatically Import the Stuff I Need.
Every time I wander off into Ruby or Python, I end up coming back a little bit sad. Such nice languages, but the tools just can't hang.
Since hard drive space is cheap, I have different Eclipse installations for each language I use. This seems to cut down on plugin hell and slowdowns that people often complain about.
I can't comment on how good Eclipse is actually for web development, but for embedded Javascript scripting it works splendidly. Most precisely, it lets me zoom in and out of abstractions in a large project with multiple files with ease.
JetBrains WebStorm. While not really up to level of some of the better IDEs for other languages, it's probably the best JavaScript IDE currently available.
I really wonder how much time the author has spent with a modern IDE like Eclipse. For Java development, it really does have all of the features on his wish list, or something equivalent.
That was my thought too, Eclipse JDT at least is pretty impressive, some of the other plugins not so much. I was surprised to see IntelliJ mentioned as supposedly good too, I wouldn't say it's terrible, but I find it somewhat lacking in comparison to JDT.
Indeed, Eclipse for Java is just unbeatable. I just recently started using it for C++ and it also works surprisingly well. I find myself much more productive than in a text editor, especially in a large code base I didn't know before.
but languages are written in text and IDEs are basically text editors with a few extra features.
Not necessarily. The text editor features of most Smalltalk IDEs were pretty gimpy, but the text editing was never the exciting part! It's the whole environment being like iPython on steroids everywhere and all the time.
I've worked on java projects in both vi and eclipse. While ramp up on larger projects generally takes longer in vi, I feel like the payoff is worth it. What eclipse does for you in suggesting classes become memory muscle pretty quickly in vi. At which point, eclipses slowness is just crippling. If someone could make an IDE that was fast, and could figure out the project completely from my build tool without nudging, I might enjoy it. I just don't see how a more feature rich IDE is going to be faster and less buggy.
a lot of it has to do with programmers bitching about bad programs way more than users of those other programs. he has obviously never used protools or any audio editing program. protools is one of the best out there, but it is still awful, as are the rest of them. it took digidesign almost a year to add support for leopard and then they charged $150 for the update.
"At least I’ve heard IntelliJ IDEA is great, I’ve never used it."
That's like complaining about image manipulation software when all you've used is GIMP.
I'm a vim guy from way back, I always avoided big IDEs because of the bloat. IDEA is the first I've tried where the IDE doesn't feel like it's getting in the way, and actually makes the job easier.
"click a little [+] next to predict(p) and see it right there inline, with its argument replaced by p? Oh, that’s how that works, cool, [-]."
While I like this idea, I fail to see how having to use my mouse to navigate to and click the +/- would be more useful than using a keyboard shortcut to jump to definition..
I also think I would find this "inlining" quite confusing in an OOP language, where the "predict" method was a member of an object instance, containing code that referenced this/self.
I would love this feature, because I regularly have to deal with OOP code that is constructed solely of functions that are about three lines long. This often swaps method-level complexity for systemic complexity, and it drives me nuts. Having to look back through ten different files to find out how a method does what it does as well as what it does (i.e beyond what you find in your average comment) is infuriating. Being able to inline method definitions would be really helpful, if it could be done in a way that didn't cause breakages around 'this', and so on.
The point is locality: if you jump to the definition, then the site of the call is no longer on screen. Being able to easily bring up the definition or at least the prototype and a one-line description without losing the current context would be quite useful.
Google's App Inventor prevents you from making syntax errors, since you use code blocks instead of writing source code. Actually, if you use code templates that any modern IDE has, it helps you to avoid most syntax errors as well...
just learn to use a good text editor like emacs or vim, I use emacs and it fits me like glow whatever text based activity I am currently engaging at that moment.
And yes there can be better IDEs but I wont be using them.
Take the photo clone tool mentioned. Why does that work? Because human vision is lossy and forgiving - the difference between 95% correct and 100% correct is around 5%, depending on how closely the person is looking at it.
Programming on the other hand... code is either 0% correct or 100% correct. Anything less than 100% correct is broken.
Media is also far less complex. How do I know if my video is correct? I simply play it. If there any errors, they are all possible to detect in the minutes it takes to play the video.
There will never be an equivalent for code. You can't visualize an arbitrary algorithm over all its inputs - that's undecidable. It's hard enough to see what a function in three variables does, let alone quicksort.
The best we can do is have editors for specific tasks. GUI editors, markup editors, etc. For everything else, code is the only option. We can patch the awkwardness with better languages and better IDEs, but it will never, ever, go away.