I've been thinking about a design for a new type of development environment, particularly suited for extreme hacking. The core concept is to centrally host all of the necessary tools of the development process, from documentation to code to even compilers, interpreters, and application servers.

It starts off with a database for assets. The database keeps everything revision controlled, while also providing a high degree of searchability across all of the assets. I'm not even talking about storing full source files in an nvarchar(max) field in some "documents" table. I mean that some level of the lexical parse tree itself is stored in the database. Query to see all of the classes within a namespace, query to see what methods a specific class has, query to see the parameters on a method. A requirements "document" isn't an actual document (I freaking hate MS Word requirements documents), it's a formatted query output for requirements elements filtered by project parameters.

You'd also link assets conceptually, this requirement asset here is fulfilled by this code revision set there. Team Foundation Server sort of does something like this, but it's pretty clunky. Basically, you're already linking parameters to methods to classes to namespaces to projects, so why not also link together code sets to tests to requirements to projects? This is a bi-directional graph, not a tree. It should be easy to query to see who has worked on what, what parts of code are or are not covered by error-tickets/requirements/tests, what requirements haven't been implemented yet, etc.

That's storage. Next is editing. By default, every editor is a collaborative editor, much in the same way as EtherPad (though EtherPad doesn't have syntax highlighting, so it's not immediately appropriate). CodePad is also interesting in this regard; the collaboration comes from editing revisions stored on the server (more on CodePad in a minute). I doubt that people would often work collaboratively, but the option is there to do it. This doesn't have to be a browser-based app, but it would help. Some sort of thin-client, Web deployment model would be used. Editors for as many file types (well, MIME types, there aren't any files anymore) as possible.

Just like CodePad does, compile the code on the server. With a large enough cluster we can be doing real-time builds on the project on every commit. We have our test cases here, so run the test suites and report on the pass/fail rate. Binaries automatically go to a download page. Web applications automatically get deployed, either to a remote server or hosted in the DevEnv.

And that's pretty much it. One extremely integrated, vertical stack for all development needs. Some of this is doable by just doing SSH sessions into a terminal somewhere. The database-driven asset-tracking is a big motivator for me, I think tying projects to file-systems is quickly becoming unwieldy. I also think it could make certain high-level revision control a lot easier.

I like it. Im also not too fond of the OS-dependent-file-system+raw-text-files paradigm.

Regardless of online thingies (which I certainly see the merits of!), what id like to see is a sort of mindmap-like editing of code. All classes and methods and scopes are objects in some visual tree that can be visualized and searched in several more sophisticated ways. Whole functions and classes can be dragged and dropped into different namespaces. Information of secondary relevance could be optionally displayed or edited in a popup window, such as 'inline' keywords or other compiler hints. Order of function arguments can be reorganized by dragging and dropping, keywords toggled with checkboxes. You get the picture.

I think that this would improve intellisense-type feedback, reduce boilerplateish characters, could somewhat reduce compile times, and overall provide a much cleaner and structured way to organize code, allowing you to see more of what you are looking for at the moment, and less of what you dont.

Maybe something like that exists already, but not that I know of. Perhaps that means there is a good reason it doesnt? The first reason as to why that pops up in my mind is the 'plain text editors for the win' mindset prevalent among coders, but I might be missing something.

My problem with tool stacks like this is that they tend to become very centered on a particular workflow, and for projects which don't conform to that workflow, they tend to be more or less useless. A lot of things get in the way rather than being helpful, and so on.

For instance, this would never work on our current game project. Integrated asset management would be cool, but we use a dozen different tools to do content creation, and each one would have to be linked to the tool stack somehow - messy. Holding the parse tree in data would be nice for doing things like code introspection and so on, but in C++ it just isn't practical; the guys at Microsoft have been trying for almost fifteen years to get Intellisense to work nicely with C++, and still haven't quite pulled it off. (Visual Assist X is a nice improvement but also lacks; it often totally breaks on larger projects, for instance.) Even if you forgo realtime updates of the code in the DB-side parse tree, you're talking a lot of data manipulation every time someone makes a commit. You would really need first-class support from the compiler to do this.

I'm still out on collaborative editing. My experiences with it have been overwhelmingly negative. Even in pair programming type scenarios, I find it far more effective to have one person editing a file at a time, or do traditional merge-style updates like virtually every VCS out there does anyways.

The continuous integration potential for doing live/realtime compiles is tempting, but again, may not be practical for most environments. We can't afford the server space to be compiling our entire game codebase on the fly; even the nightly builds take close to an hour to run. Combine that with asset dependencies and you're in for a nightmare - we constantly have situations where code and asset updates have to occur in tandem or the game will barf (sometimes more gracefully than others).


I'm all for pushing forward development paradigms, and I've never been a particularly huge fan of the file-system metaphor anyways; but IMHO there's a lot more work involved in making this truly applicable to the general case than it might seem.

I don't know if it's actually useful - who can tell until it's made - but it's certainly interesting. Sounds like the kind of research project someone might do at Google, since they're obsessed with getting everything onto the web (or cloud if we must use buzzwords).

Do Google open-source enough of their docs platform you could build a code-editor on top?

N Brain? It sounds similar.

If this were a hosted service, consider the following:

#include </dev/random>

At which point the question becomes, how much would it cost per cycle/hour/something.
Or how much it would require to host locally.

Quote:

Original post by ApochPiQ For instance, this would never work on our current game project. Integrated asset management would be cool, but we use a dozen different tools to do content creation, and each one would have to be linked to the tool stack somehow - messy. Holding the parse tree in data would be nice for doing things like code introspection and so on, but in C++ it just isn't practical; the guys at Microsoft have been trying for almost fifteen years to get Intellisense to work nicely with C++, and still haven't quite pulled it off. (Visual Assist X is a nice improvement but also lacks; it often totally breaks on larger projects, for instance.) Even if you forgo realtime updates of the code in the DB-side parse tree, you're talking a lot of data manipulation every time someone makes a commit. You would really need first-class support from the compiler to do this.

Agreed; I am no compiler programmer, but trying to make this work with C++ for instance seems like a nightmare to me. It should tightly integrate with the compiler to work as advertized.

Quote:

Original post by ApochPiQ I'm all for pushing forward development paradigms, and I've never been a particularly huge fan of the file-system metaphor anyways; but IMHO there's a lot more work involved in making this truly applicable to the general case than it might seem.

Well, my personal development strategy is basically "code for the current situation, generalize only if you have to, but don't delay generalizing when it becomes clear." So something like this would start off being very suited to building Web applications, but not necessarily games (or anything C++, for the reasons you mention).

I think just having an online editor for Python/Ruby/Lisp/Whatever scripts, stored in that aforementioned Documents table with the NVARCHAR(MAX) column with auto-site-deploy would be incredibly useful. That's a minimum level of functionality that I think is doable in a short amount of time and can be iterated towards my goal from there.

It would be necessary for a start anyway, because I'm not entirely certain what it would take to go from an in-database parse tree to something that could be fed into an existing compiler, but if you're storing just a representation of the files that would go to the compiler anyway, that's not going to be hard to support off-the-bat.

Along the way, who knows what happens? Maybe this develops into its own set of programming languages. Maybe you don't do C++, you do a language largely compatible with C++ that fixes the legacy cruft that makes C++ take an hour to compile. And then eventually you don't need the C++ compatibility. C and C++ are languages that are designed around files, Java is a language that is fundamentally tied to a file system, if you remove that fundamental assumption, the languages necessarily have to change.

Languages with a REPL are already treating the source code as something more than just a stream of bytes sitting on a disk, that's just the backup format. How were programs written before the abstraction of a file system was invented? Punch cards, dip switches. Improving our storage abstraction is likely to improve our languages to the point that we won't even consider using the previous generation.

But I do know that it won't all happen at once. So I think the minimum set is a good start on a road to the Final Solution (Files System Genocide).

Quote:

Original post by Sirisian N Brain? It sounds similar.

I can tell you from experience that the example they demonstrate here is never going to happen. There is a constant fear of stepping on the other editors' toes. Deferring to other editors can get so bad that deadlock occurs.

That said, I do think there is some value to shared-view editors. As whiteboards in tandem with voice chat, they are great. In such a setup, there can be a clear delineation and passing of the editor role, avoiding the toe stepping as well as the deference dead lock. And on rare occasions it can be helpful to have multiple people looking at the same code at the same time.

Now, there is typically far too much to do with a project to have this be the norm. Normally, people are going to work by themselves on separate parts of the project. This is fine--a collaborative editor with only one user is no different than a regular editor.

All the visual programming tools I can google are special propose 'programming' languages. It really doesnt seem like such a radical idea to add it to a general purpose language. Its essentially just 'strong structuring', no more a controversial preference than strong typing. Still, seems like im missing something. Creating an editor in this format that could export to c# or python is nearly trivial, but translating intellisense/compiler feedback back seems prohibitively complicated though. Perhaps thats why it doesnt happen?