Tag: programming

Lenya

Late last week, Michael sent out our proposal to donate the wyona code base (renamed lenya) to apache. it has been a very interesting discussion since, and so far it is looking good. in any event, it feels great to enter a new community.
2003-04-08: This Thursday and Friday (April 10th and 11th), Michael and I will be at the office of Q42 in Den Haag. We will do a “Lenya Mini Sprint” there and the guys from Q42 will help us improve the Lenya User Interface. what is a sprint? We will also try to integrate Xopus 2.0.0.8 (which was the last Open Source version) and try to integrate their newest Xopus version (which is not Open Source). At the end of Friday there will be a reception at Q42’s office. Please let us know if you want to meet us in Den Haag or join the party on Friday night.
2004-04-25: the apache lenya community is organizing a sprint on may 14 and 15 in zurich, switzerland. we will focus on repository integration, but people with other interests are also welcome, of course. please sign up on the wiki (and watch the wiki for updates) if you would like to attend.
2003-04-11: just received word that the university of reunion will be using Apache Lenya for their site. here is an offer: pay for the flight and accommodation, and i will help setting up Apache Lenya 🙂

2003-05-13: turns out my work for the past couple weeks has been totally in vain. what started as (yet another) attempt to write a “clean” publication for apache lenya resulted in another publication with hacks. in retrospect, DOH of course, because each publication has its quirks that you only run into when you develop it. the time would have been much better spent tightening up the existing code base. instead, we broke code that was working perfectly with stupid changes, like changing the root element of aggregates. i have no one to blame but myself for this, but i will make damn sure we don’t waste any more time like this. i wonder why software “engineering” has to be so painful. we need a regimen that is more focussed on the bottom line to do away with such nonsense.
2003-05-23: i’m reading through the Apache mirroring FAQ today to figure out how to do a release. i’m pretty excited, luck permitting, Apache Lenya 1.0rc1 should hit the mirrors later today.
update. we are live.
2003-05-27: here is my newest try at shaping up my marketese speak. some gems:

  • content delivery
  • technologists
  • preeminent platform
  • future-proof

2003-06-12: son-of unipublic. this is (to my knowledge) the first independent deployment of apache lenya. very cool. now, when is their opening party?

2003-07-28:
U src/webapp/lenya/pubs/default/lenya/xslt/authoring/create.xsl
failed due to an internal error (took 0:18.754)
it’s high time for CVS to roll over and die. it’s numerous deficiencies are beginning to seriously piss me off.
2003-09-10: bill humphries is just demoing the apple intranet. it is based heavily on PHP and uses the concept of page envelope we came up with for lenya. very cool stuff.
2003-09-20: michael and myself paid a visit to bill at apple today to demo lenya. unfortunately i was not allowed to take pictures on the campus.
2003-11-15: i’m tooling away in las vegas, at the apachecon hackathon, trying to get apache lenya 1.2 out the door. meanwhile, my landlord is having fun with tools of his own. too much fun.

2003-11-28:

NZZ Online and Computerworld are 2 out of the 3 winners of the SiteAngel Trophy, a benchmark comparing performance and availability of swiss media websites. Both web sites are built on Apache Lenya, the open source Content Management System (CMS) originally developed by Wyona Inc. the leading swiss specialist for open source CMS.

computerworld was my first project at wyona.
2003-12-22: michi will represent lenya on a panel at iex that also features representatives from day, interwoven, obtree and red dot. michi will also be speaking at a pre-conference open source forum.
2004-01-30: i want to bring these 2 interesting workshops to your attention. They will be held on Wednesday, February 18 at University of Berne as part of the LOTS (Let’s Open the Source) event. Registration is open until February 16th
Supersonic Tour of Apache Cocoon
Speaker: Bertrand Delacretaz
This fast-paced tutorial gives a quick overview of the Apache Cocoon web applications framework, with a more detailed view of the more mainstream components of this framework: the multi-channel publishing subsystem, the Flow layer and the Cocoon Forms (aka Woody) components. By studying the inner workings of example pipelines and small applications, we will learn how to combine these components to take advantage of the massive amount of functionality offered by Apache Cocoon’s rich set of components.
Focused samples will help us understand what goes where and which components must be studied to be productive with Apache Cocoon.
Attendees are welcome to bring their laptops with the latest release of Apache Cocoon installed, for hands-on exploration during the tutorial, but the packed agenda does not allow for any help with installation problems during the presentation.
Apache Lenya Workshop
Speakers: Gregor J. Rothfuss, Andreas Hartmann
Wyona AG
Familiarize yourself with the features and architecture of Apache Lenya, the Content Management System from the Apache Software Foundation. The workshop will outline functionality by demonstrating projects (University of Zurich, NZZ) that were built using Apache Lenya. An architectural overview, building on Apache Cocoon (which is covered in a separate workshop) will be followed by lessons learned from our customer projects. Participants will learn valuable lessons for content acquisition, content architecture and cms customization, and there will be plenty of time to ask questions. The duration of the workshop is 3.5 hours.
2004-03-21: i’m always learning about new cool tools that ship with cocoon. today, i had a look at the instrumentation support. it allows you to analyze runtime behavior of lenya to spot troublesome areas. you can check memory usage, objects in cache, and many more.
2004-04-11: apache lenya now supports i18n, and conversely, we are accepting translations for the user interface. i am wondering if these should eventually go into their own repository? if the xaraya project is any indication, translations can quickly outgrow the code (in numbers of files.) openlogging reports that the xaraya translations repository is now the second largest bitkeeper repository, larger than the linux kernel. scary. 🙂
2004-05-11:

Cofax 1% (3)
DBPrism 0% (1)
Lenya 23% (40)
Magnolia 18% (31)
MMBase 1% (3)
Nukes 6% (11)
OpenCMS 7% (12)
RedHat CCM 0% (1)
Slide 2% (4)
Other (please comment) 2% (4)
Don’t know, I’ve never used any of them! 25% (43)
OpenEdit 2% (4)
InfoGlue 7% (12)

2004-06-07: it’s nice to get press even if it is not as clued-in as it could be:

Version 2.1.5 fixes numerous issues with the framework and is available for Unix and Windows environments. Stefano Mazzocchi who just last week was elected to the Apache Software Foundation’s Board of Directors, suggests that v2.1.5 is the most stable version yet released. In general, the community touts Cocoon’s XML framework as the “Web glue for your Web application development needs.” 1 of the direct Apache Cocoon sponsored projects, currently in incubation (not a fully ASF supported project) that relies on Cocoon is a Web content management system known as Lenya. The latest Cocoon release fixes at least 1 critical bug that will make Lenya more functional, according to Lenya committer Gregor Rothfuss. “We were quite happy to see it resolved”. The next Lenya CMS release is scheduled for release in the next 2 weeks.

you heard it, people. lenya 1.2 should be out soon 🙂
2004-08-03: with that start of the olympics, it’s a good time to mention that the national olympic committee of afghanistan runs on lenya.
2004-09-23: The board of the Apache Software Foundation passed the resolution to make Apache Lenya a top-level project.
2004-11-22: There will be a boston cocoon / lenya user group meeting on december 9th at 6PM, location to be determined. i will be speaking about creating print on demand solutions with cocoon. bring your friends, and check back at the wiki for more information.
2004-12-01: received the following email in the lenya developer mailing list moderation queue today:

Dear Lenya,
I am a researcher here at Vault, the career information company, and I believe that you work at Apache. I am contacting you to invite you to share your thoughts on corporate culture, recruiting, and other career topics at Apache (or another current/former employer) by filling out this brief anonymous online survey

2005-03-31: there is now a cocoon meetup group for cambridge. if you have an interest in apache cocoon or apache lenya, sign up to be informed about events in the boston area.
2005-05-21: the ASF now offers a solaris 10 zone to each pmc. not wanting to pass on a great opportunity to test drive it, i set up a lenya demo on our zone. i haven’t had a lot of time to explore yet, but some first impressions are in order: it looks like solaris 10 has a more reasonable selection of software installed, but the root shell still defaults to one that does not handle arrow keys, or sensible tab completion.. what’s up with that? i look forward most to exploring dtrace, especially in conjunction with java.
2005-06-02: The Apache Software Foundation is a proud partner of the Google Summer of Code initiative. The Summer of Code is a program designed to introduce students to the world of Open Source Software Development and provide them with a $4500 award for completing an Open Source project before the end of Summer. The Apache Lenya project currently has 3 project proposals as part of the Summer of Code, more may be added later. For details see
Overhaul search facilities
Implement editor API
Implement workflow queries
The deadline for application is June 14th so if you are interested you need act quickly. Competition is very high for these projects, but then so are the rewards. If you would like to create a proposal for any of the above then we need to follow the following process:

- student gains an overview understanding of the technologies involved in the proposal
- student expands the initial project outline to a draft proposal text:
- description of problem
- description of a proposed solution
- benefits of the solution to the Apache community
- an approach
- milestones in delivery
- expected timeline for delivery
- description of relevant students skills
- student and mentors work together to finalize this proposal
- proposal is submitted to project community for comment/approval
- student registers the proposal with Google via their website

The Apache Lenya Team
2005-06-03: apache lenya contributor jonathan linczak was interviewed by the web standards project about his experience in implementing a standards-based site. jon is responsible for the new fully CSS-based menu in the upcoming 1.2.4 release, and has written many tutorials. people like jon really unleash the standards-based goodness inside lenya, and it’s nice to see recognition for that.
2005-06-09: Apache Lenya now has WebDAV integration, thanks to a patch by Doug Chestnut.

2006-01-26: Kudos to my former colleagues at wyona for landing wired as a customer for apache lenya. While it would of course have been nice if this had happened on my watch as COO, it’s still satisfying to know that my laying the groundwork led to such outcomes.
2006-11-01: BeCompany, a new apache lenya systems integrator.
2007-06-19: welcome to radical transparency. eventually, you’ll be able to see this sort of graph (Apache Lenya Top 10 Committers) for every working person in the world. looking busy while doing nothing is on the way out.

BNF

As i’m learning lots about static analysis, i’m re-discovering some classic computer science papers, such as the one (1977) by john backus (of BNF and FORTRAN fame):

Conventional programming languages are growing ever more enormous, but not stronger. Inherent defects at the most basic level cause them to be both fat and weak: their primitive word-at-a-time style of programming inherited from their common ancestor–the von Neumann computer, their close coupling of semantics to state transitions, their division of programming into a world of expressions and a world of statements, their inability to effectively use powerful combining forms for building new programs from existing ones, and their lack of useful mathematical properties for reasoning about programs.

i can relate to his statement that Programming languages appear to be in trouble. each new language adds new “features”, yet little changes, and we are still thinking in terms of state machines, very low-level indeed.
further down, we learn that

The models of computing systems that underlie programming languages fall into 3 classes: (a) simple operational models (e.g., Turing machines), (b) applicative models (e.g., the lambda calculus), and (c) von Neumann models (e.g., conventional computers and programming languages). Each class of models has an important difficulty: The programs of class (a) are inscrutable; class (b) models cannot save information from one program to the next; class (c) models have unusable foundations and programs that are conceptually unhelpful.

the main argument is that since traditional languages model the behavior of hardware, they are bound by its limitations:

Thus variables = storage cells; assignment statements = fetching, storing, and arithmetic; control statements = jump and test instructions.

i always hated functional languages when we studied them (they seemed less useful for interfacing with APIs, which is what most programming these days is about), but maybe i should reconsider.

big words and small talk

Innumerable pundits and programmers have pointed out the similarities between the most popular .NET languages (currently VB.NET and C#), and some like to focus on the relatively minor differences between them. S# is different. Not only is its syntax different, following the model of classic Smalltalks, but its underlying design and capabilities differ as well.
In 1999, Simmons was invited along with experts in 10 to 15 other languages, to join a then-secret project at Microsoft called Project 7 Lightning. Project 7 Lightening involved creating test implementations of these languages on the nascent .NET Framework code base to help Microsoft discover what features the framework needed to fully implement these languages in the future.

the .net platform (and by extension, mono) are getting more interesting every day. as more and more non-mainstream languages such as eiffel, scheme, haskell, ruby, python and now smalltalk get a .net port, the options increase. every self-respecting software engineer tries to learn several languages, but practicalities and a lack of a rich framework in the more research-oriented choices made this impractical in the past.
i am hopeful that we will see a day soon when there is a unified, cross-language and cross-platform class library. it wont be java, so my bets are increasingly on .net. in a perfect world, this library would incorporate CPAN, PEAR, and others. still missing: php.net

Feeling guilty about code

Russell Beattie mentions how much it sucks to feel guilty about not supporting your code:

I know it sounds horrible, but follow me on this. Once a week I get an email about some issue and I feel guilty about not updating my code or not working on improving something I published months ago (that now I’m no longer interested in). Feeling guilty sucks! Or I need to respond to help someone with an install or help figure out some random bug, or something else. Ugh! That’s not fun! That sucks! That isn’t hobby, that’s helpdesk! I mean, I like hearing from people who like my code (then again that means I have to take time to thank them for thanking me which is sort of pain) but this whole having to help people figure out something that many times you can’t even remember writing is a big PITA.

i totally understand russell’s point, i have felt the same way myself. working on open source can be much more grueling than other projects. the internet never sleeps, and popular projects have users that track you down. one way out is to declare open source software that is quickly hacked, but useful code to be under the OSS PROTOTYPE LICENSE.

We need a license that says something like “I agree, by using this code, never to bug author with any questions, comments, thanks or support-related issues after 2 months of release date when the author has moved on to other more interesting things to work on.”

right-on. heh, reminds me to update my thesis with these new developments now that i have submitted it to the MIT OSS center.

Starting from scratch is a bad idea

sometimes, it’s just not feasible to re-write software. even though most developers have an urge to do so with code they have not messed up themselves. joel spolsky argues:

..it takes 10 years to write good software.. ..re-writing software is the greatest strategic mistake you can make..

i agree strongly, especially for web projects. i’m currently trying to salvage a project that has some real messy code. however, far from throwing out everything, we try to fix it up piecewise. joel has some other good articles at his web site.

First cvs commit

well, after some weekend hacking, i have committed XML-RPC support to the postnuke CVS. unfortunately i had no time to finish it properly, so it will probably not work for now. it’s most likely a minor issue though, and maybe there is someone else interested in fixing it up. meanwhile over on bloggerdev, the discussion continues.. and some other open source projects might support this api in the future.

it will be interesting to wait for a little while and then cook up a great demo for the talk series i am preparing at zurich university.

Infosets are pipes

xml applications are infoset pipelines. henry thompson, w3c

one of the most powerful concepts in computer science is arguably abstraction. by building on abstraction, it has been somewhat possible to manage complexity. productivity advanced or new approaches are more often than not based on abstracting existing ideas and piling on them.

xml is such a story as well. like everyone and their brother i long thought xml was basically a nice way to do markup, sort of html done right. which led to the question what all the hype was about. back then i attributed the hype to the ground breaking insight that simplicity matters. other than that, i failed to see what xml could be useful for.

as it turns out, tags and markup are irrelevant. what matters are infosets, or the information that is contained in xml. with the advent of xml schema it has become possible to add another layer of abstraction to the markup. you no longer have to think of your data in terms of tags and markup, but rather in terms of its types. what does that mean?

it means you can concern yourself with the (simple / complex) types you encounter in your problem space. like the notion of address. you do not care how address is encoded, you just care about its type. xml schema allows you to extract that information out of your data. once you have such rich, structured data, you can do a lot with that.

for instance, the concept of pipes, another powerful abstraction and fundamental to the unix way. conceived by ken thompson as little programs that sequentially work on each others output, it has inspired 20 years of operating system design.

now one of the basic assumptions of the pipes idea was that the data was basically character data. enhance this concept with xml (rich, strongly typed data) and you have the foundation for a lot of new, very powerful ideas. this is what i currently understand xml to be, and what is being built on with xml protocols and ultimately the xml processing model.

(heavily inspired by this keynote talk by henry thompson)

Collaborative development

We build WYSIWYG editors for sharing richly-annotated source; we use glass TTY editors (vi, emacs) to hammer on flat ASCII files in which most of the metadata (comments) is completely unstructured.

i just discovered jon udells very interesting newsgroups. some nice quotes about software development in the web era:

greg wilson on why xml-style coding might not catch on:

Many programmers would rather change the way the global economy works than change the way they brace and indent their C code. Many also program as an end in itself (even when they’re being paid to do so). They don’t see an advantage in switching to a format that looks odd (compared to what they’ve been weaned on), and whose primary benefits are team-oriented (lower long-term maintenance costs, easier for newcomers to navigate the code, etc.).

very very sad. i’d gladly switch to tools that allow me to capture my intent on a higher semantical level. however there is apparently hope:

greg again:

I am hoping that as old programmers die off, younger ones will start bringing tools that they’re used to using in other contexts into the coding arena. For example, the proportion of programmers using IDEs instead of legacy command line tools (Emacs + gdb, makefiles, etc.) seems to be increasing. I believe this is because students are introduced to both while they’re still impressionable, so they can choose without worrying about the cost of change.

on the need to include communication in the process:

Finally, LP systems only addressed the problem of “static” communication — I write a doc, you read it, information flows one way. This only addresses the needs of big corporate dev environments, where audit trails and 20-year life cycles are an issue. I’ll bet most of the people reading this group need something much more fluid to sustain their day-to-day work. Even those of us in our late thirties are now used to the “dynamic” 2-way threaded ongoing communication of newsgroups like this. I’ve watched developers in their 20s use Messenger to throw around hasty sketches of data structures and snippets of code while chatting; offer them an IDE that does this, and their reaction is likely to be, “Well, duh, about time.”

i for one would love to integrate jabber with my ide. given a decent xslt, these conversations could be nicely integrated with the source to provide more clues about the code.

sourceforge is a first attempt at such a highly integrated ide that tries to embrace group ware. finally there seems to be a very good book by karl fogel on Open Source Development With CVS that will hopefully be on my bookshelf one day.

soap on apache

i am digging into soap now, and it’s a mess who would have thought that installing soap on apache were such an undertaking? since soap for apache is basically donated java code from ibm, it of course relies on the entire java framework for apache being present.

1 thing i always hated about java was the incredible mess it had with all its 100s of directories, 10s of config files, $CLASSPATH and so on. now while many open source projects have not exactly been known for good documentation (perhaps this “lesser task” is beneath self-declared hackers) it seems to be worse with open source projects in java.

anyway i did not have the nerve after a full days work to read up on all these arcane details i frankly could care less about. so no soap for apache today.

competing with .net does not just mean whipping together a soap stack and create some bindings for php, perl and python. web services is about leveraging infrastructure to get quick results. if you have to fiddle around with configuration details you could just as well skip web services. there is a strong need for a ready to run soap package where everything is already neatly configured and integrated. you should be able to have a hello world up and running in minutes. that’s what they deliver today on the microsoft side of the fence.