Birth of a New Product-4

Time for Rethinking

by Jack Krupansky

How embarrassing.  It's been a full year since I decided to get 
serious about doing a new product and I still don't have one.  I'm not 
even close.  In truth, I don't even have a good start on one.  I've 
made a lot of progress in the last year, focusing on software agents, 
but the net result is more in the way of concepts, ideas, and market 
awareness than actual, tangible product.  And I've certainly 
accumulated a mountain of excuses, rationales, and good reasons for my 
lack of a product.  Priorities, especially money, are also a big part 
of the problem.  The only really good news is that my target market is 
still very undeveloped.  So much for having completed the "final idea 
selection" phase.

Life Goes On

Since my last article (August 1997) I've certainly done a lot of 
struggling.  I lived for a couple of months in New Hampshire while 
trying to line up consulting work.  My income from my old consulting 
client was reduced to a mere retainer for about twenty hours of work 
per month.  I just wasn't able to find any work that fit my 
experience.  This reduced income apparently caused my credit card debt 
to balloon.  In late October I moved to Cambridge, Massachusetts and 
found a small business that had a pressing development project that 
needed my skills.  Unfortunately, it was only about two months of work 
and at a significantly lower hourly rate that I've grown accustomed 
to.  In fact, it was less money than I was getting five years ago!  
So, my credit card debt didn't shrink very much for that period of 
time.  But I was hungry and even such a bare bone looked quite 
appetizing at the time.  January was another month of down time and of 
course increased credit card debt.  Spending five days in New York 
City for New Years didn't help my finances much either.  Luckily I 
still had that small retainer from last year's client, so I was able 
to barely stay afloat.

Needless to say, I've been trying to cut my expenses as much as 
possible.  I got rid of my New York remote call forwarding line, but 
retained my 800 number and answering service.  I've tried to minimize 
long distance phone calls and make more use of email.  With an 
apartment in Cambridge and a client within walking distance, I'm able 
to maintain a car-free lifestyle.  I'm much happier living in an area 
where I can use my feet as my primary daily transportation.  My 
apartment is more expensive than I'd like, but my net living expenses 
are the lowest they've been in a few years.

The lease on my apartment runs out at the end of August, so I guess 
it's getting to be time to move on.  Cambridge and Boston have a lot 
to offer, but not enough to keep me here.  I'd really like to live in 
Washington, D.C., but with my gigantic credit card debt, I have to 
give my first priority to living where I can get top-dollar 
consulting.  But since it's only April, it's really too soon to tell 
how things are going to shape up in the fall.  My current consulting 
is going very well, but it could easily run out real soon.  Just the 
other day in a meeting the phrases "budget review" and "hiring freeze" 
were dropped like little bombs.  It could go either way.  Consultants 
can be the first to go, but sometimes we give management more short-
term flexibility and control.  I suspect that they'll want (actually, 
need) me to stay on, but I know that I can't count on it.

One of the nice things about my new client is that that don't even 
care whether I work on-site or at home.  Several of the people I work 
with work at home one or two days a week.  So if I survive the budget 
review for the next four months, then I have a good chance of being 
able to work remotely (like from Washington.)  By the summer, I'll 
have enough experience with my client that I really won't need very 
much daily or weekly contact, so I might be able to set things up so 
that I come in every couple of weeks for "contact."  It still would be 
good for me to try to line up some consulting work in the D.C. area, 
but I just don't have many contacts down there.  I may have to stay in 
the Boston area to get the kind of work that will let me pay down my 
credit card debt.  But, being the nomad that I am, I'll go wherever 
the money is.  Fortunately, my new product ideas don't have any 
geographic requirements.

Some Progress

Back in August I had thought I was going to focus on a simple remote 
software agent subsystem as my product.  I even started writing some 
code to gain experience with "socket programming" for the Internet.  
That went quite well and I quickly had a little program running that 
would allow one system to send a small Liana program to the other 
system to execute and return results using sockets and a simple 
protocol that I developed.

Next I decided to do some due diligence using the Internet.  I started 
using the AltaVista search engine and was overwhelmed my the thousands 
of references to "software agents."  After being depressed for a 
couple of weeks I finally came up with a scheme for attacking this 
mountain of hits, all five thousand of them.  I started using 
AltaVista's date range feature so that I could slice through the 
mountain a few days at a time.  I was determined to find out what's 
really out there, by looking at the raw data myself rather than trust 
experts and analysts who filter their own opinions with their own 
biases.  There was a lot of interesting material out there along with 
a lot of junk.  But it's too mind-numbing to sit and visit all those 
pages.  I could only stand it for a few hours a week.  It took me from 
August till February to visit all those pages, but I did it.  I 
accumulated my own list of URLs for the pages and sites that I deemed 
worthwhile.  That list came to about a thousand URLs or about only one 
out of every five that the search engine reported.

Of course I could spend a lot of time trying to keep up as new pages 
show up, but I noticed that there was a lot of repetition and not a 
lot of truly new material in recent months.  I subscribe to the 
"Agents Mailing List" which is an informal email system that let's you 
keep up on what's going on related to agents.  This includes 
conferences, opinions on various agent tools, and wild debates on the 
definitions of terms like agent, intelligence, autonomy, and mobility.

Due Diligence

My research showed that there was a lot more going on in the field of 
agents than I was aware of.  The popular trade press tells you about 
only the tip of the iceberg, such as General Magic's TeleScript and 
Odyssey, Pattie Maes at the MIT Media Lab and Firefly, IBM Jaglets, 
and a few others.  But there are many others, including both 
commercial products and leading edge research projects.  Certainly a 
lot of the efforts focus on Java, but there are far more projects that 
use more exotic programming languages and tools.  Agents figure 
prominently in the field of artificial intelligence with sub-fields 
such as DAI (Distributed Artificial Intelligence), MAS (Multi-Agent 
Systems), and CIA (Computer Interface Agents.)  And a lot of work has 
been done in the area of "Internet robots" or softbots.  Many web 
sites now offer some form of email alert for news, stocks, special 
offers, etc.

There's also a bit of premature standardization going on.  There's 
something in Europe called FIPA, but my inclination is that all the 
current technology is too cumbersome and poorly thought out to matter 
for the long run.  The results of their efforts will be too cumbersome 
(or vague) for average developers to use productively, so cleaner, 
simpler, more elegant solutions will easily be able to overtake any 
Titanic-like standards heading out into a future filled with lots of 
technology icebergs.

Second Thoughts

My original idea was to focus on sending small, simple programs from 
one system to another.  Others have already done this many times, but 
there are many problems with this concept, such as security, 
infrastructure requirements, competing protocol standards, variety of 
system architectures, technical problems with Java, and a general lack 
of a good paradigm for how developers would actually use remote agents 
even if all the other problems went away.

As fascinating as it is, I have been trying to avoid getting mired 
down in the swamp called Artificial Intelligence.  Agents and 
intelligence are tightly bound for a lot of researchers.  Dumb agents 
are "simpler", but are difficult to construct, debug, test, deploy, 
control, and maintain.  A simple toolkit is not the answer, but 
waiting for the experts to agree on a definition of intelligence isn't 
either.  I think the answer lies in coming up with a number of fairly 
narrow but well thought out paradigms that are embodied in parallel 
sets of very high level protocols coupled with very high level code 
libraries.  The agent developers can then concentrate on what's 
important to them, knowing that when they make mistakes this very high 
level agent infrastructure (call it an Agent Operating System) will 
protect them and the rest of the world.  Such a very high level 
approach is especially needed since one of the prime applications for 
agents is electronic commerce with "real" money.

But today, agent developers struggle just trying to get the various 
tool kits to work at all.  Most of them seem to be in perpetual "beta" 
or unsupported mode.  Serious researchers recognize that "trust" is an 
absolute requirement for agents to be workable in the real world 
(especially when electronic commerce is involved.)  But they also 
acknowledge that trust is a research topic with no solution readily at 
hand.

Mobility of agents seemed so obvious at first.  It was the heart of 
General Magic's TeleScript and the focus of a number of Java-based 
agent toolkits.  Researchers focused on how to allow a program to 
encapsulate its code and data, send it across the network to a waiting 
server, de-encapsulate the code and data, and continue running.  This 
is a non-trivial problem, but is in fact one of the smaller problems 
to be solved before agents can achieve widespread acceptance.  Sure we 
can have all these agents move around.  Now how exactly do you get 
results back?  How can a human user control a running agent?  How can 
agents talk to each other?  How can I know that my agent is now out of 
control?  And what can I do about it?  Agents are really just 
programs, so they'll have bugs, so how can they be debugged?  Does 
anyone really believe that your average system administrator is going 
to voluntarily allow foreign agents to run on their systems?  These 
are just a few of the questions.  At this stage, the list of questions 
is much longer than the list of answers.  And each step forward raises 
even more questions.

It turns out that much of what people want to do with agents can be 
done using remote access rather than trying to move code across the 
network.  Mobility made sense in a world or very skinny pipes 
(remember 9600 baud?), but in a world of fat pipes, those reasons are 
far less compelling.

These and many more issues that I've run into and struggled with over 
the last seven months have definitely slowed me down and depressed me, 
but that's par for the course.  I'm actually starting to get more 
energized by all these issues.  The sheer volume of issues will serve 
to slow everyone else down.  If I'm clever about picking the fights 
that I know I can win and sidestep the deeper parts of the swamp then 
I can do better than most.

Other Excuses

Different tasks require different energy levels.  With all the 
financial pressures of the past year, my energy level has been much 
more suited for doing casual research and contemplating ideas.  Add to 
that my depression over my lack of progress and the hard problems of 
agent technology. Designing software takes a lot of energy and a 
special kind of energy that I don't always have.  When I have it 
things go real fast and when I don't have it things go excruciatingly 
slow.  But sometimes that works out for the best.  If I had actually 
implemented the world's best mobile agent toolkit, what good would it 
do me or anyone else?  Sure I could sell (or at least give away) a few 
hundred copies to the lunatic fringe.  But even I don't want that 
technology today.  Maybe I should count myself lucky that I was able 
to keep that idea from becoming a real product.  The bottom line is to 
go with the flow.

One of my favorite excuses is that the market for agents still isn't 
very well developed.  It's true, but so what.  After all, I want to be 
on the leading edge and not following someone else's lead.  As soon as 
I zero in on the ideal angle and marshal an appropriate level of 
energy, then just get out of my way!  Ultimately, the advice I should 
give myself is quite simple:  Just do it!

Another excuse is that most IT organizations at big companies are so 
overwhelmed by current technologies (Unix, NT, client/server, web, 
Java, thin client) and looming problems (Y2K, the Euro, increased 
competition, increased pressure from Wall Street) that they're just 
not ready to take on yet another new technology.  They certainly need 
tools to enhance productivity with existing technologies.  But a whole 
new paradigm for distributed computing?  Forget it.  But there still 
is the lunatic fringe and there are lots of serious projects that are 
chartered to push the envelope of the Internet and distributed 
computing.

My other favorite excuse is that maybe I'm just getting too old for 
this kind of thing and I should just leave all this newfangled agent 
and Internet stuff to the energetic, younger generation.  But there's 
no way that I could bring myself to admit that kind of defeat!

I've also found myself spending way too much time managing my 
retirement plan.  In past years it was too small an amount of money to 
attract much of my attention and I had a long-term buy and hold 
strategy.  But it's now growing faster than my total consulting 
income.  Unfortunately, last year my self-directed plan which is 100% 
high-tech under-performed the S&P 500 index by 5%, primarily because I 
had a few dogs that I should have sold early in the year.

I didn't even have an annual return goal.  Now I have set a goal of 
achieving a 30% annual return (which I've already exceeded in just 
three months this year.)  Stocks will stay in my portfolio only if 
they show long-term prospects of  earnings growth in that range.  I 
also give a lot of weight to the quality and focus of a company's 
management team.  Dell stays, but Compaq may go, although it's so 
beaten down now that it may give a healthy short-term return by the 
end of the year.

I'm on the lookout for smaller, niche stocks that have the potential 
for dramatic earnings growth over the next few years.  And that would 
be great candidates for buyout by Microsoft or one of the 
communications or networking companies.  But I'm still staying with 
high-tech, since that's what I understand the best.  No Yahoo or 
Amazon, but I just wish someone could convince me one way or another 
whether I should continue my long-term holding of America Online 
(which I bought because Goldman Sachs gave it a high rating) which has 
tripled for me in just over two years and now has real earnings but 
otherwise seems to be skyrocketing just because it's billed as an 
"Internet stock."

I'm also unsure what to do with my WorldCom holdings (which I ended up 
with by buying UUNet when I heard that Microsoft had invested in it) 
since I don't understand exactly how these hybrids of traditional 
telcos and leading edge fiber/packet technologies are going to make 
money (my 30% earnings growth goal again.)

I don't want to be a trader, but I have my annual retirement plan 
contribution coming up over the next five months, so I have to worry 
about where to place my long-term bets.  George Gilder seems to think 
leading-edge communications is the way to go, but I'm still focused on 
Dell/Cisco-like earnings growth.  Feel free to email me all your stock 
tips and advice.  And don't forget to alert me a few days in advance 
of "the" big correction.

New Direction

I don't yet have a new plan of attack, but through the fog I think I'm 
beginning to make out a few details.  I'm almost ready to give up on 
the mobility issue and stick with remote access.  Java is here to stay 
and already has a lot of features for operating on the Internet.  
Higher-level, application domain-specific protocols are probably the 
way to go.  Secure sockets and encryption are a win and here today to 
address security issues.  E-commerce is hot.  User-friendly alerts via 
email are hot.  Email might be an ideal way for users to talk to and 
control their agents.  Network traffic is troublesome even with lots 
of fat pipes, so non-real-time communications such as email are a win.

One of the other reasons for mobility of agents was for users who are 
not connected all the time.  But I suspect that a better solution 
would be to have agent servers which are a place where you can 
"submit" your agent to let it run and remotely access network 
resources while you are doing other things or not even connected to 
the network.  Kind of an updated version of "batch" processing.  
Remember RJE (Remote Job Entry)?  Maybe Bill and Strauss and Howe in 
the Fourth Turning are right and history does have cycles!

Another area is that of agent name servers (ANS) which are really just 
directory services so an agent can access a network resource by it's 
logical name rather than with traditional Internet features such as 
domain names, port numbers, and IP addresses.  I have some ideas for 
what I'd like to see.  If my agent needs stock quotes, I'd like to not 
have to worry about selecting a specific source of quotes, but rather 
to specify the requirements (delayed, real-time, how much I'll pay, 
guaranteed response time, etc.) and let the name server match my agent 
up with an available network resource.

There's actually some potential with my new client.  They offer a 
Internet search engine (www.northernlight.com) and say that they were 
originally thinking of offering some kind of agent feature.  I may be 
able to work out some kind of deal with them and develop some kind of 
agent technology that works in conjunction with their search engine.

Since I haven't been feeling energetic enough to design and code an 
extremely complex system architecture, I have two choices:  wait until 
I do have enough energy or just come up with a simpler, more 
streamlined design.  I lean towards the latter.  I have always found 
that the intellectual challenge of producing a clean, tight, elegant 
solution more than makes up for a lack of overall energy.  Having a 
zillion features to implement (and test) is just too draining.  
Besides, I'm getting too tired for those kinds of complex solutions;  
I'll leave them for the up-and-coming Generation-X Internet-babies.

I think simple, elegant products are one of the best ways for those of 
us with lots of experience to provide the youngsters with good 
examples of what they should be striving for.  I don't want to measure 
my ability in millions of lines of code I can produce per decade, but 
in how few lines of code my solutions require.  It's just amazing how 
rapidly the complexity of a product can spiral downwards once a single 
individual is able to continuously grasp all of its details in their 
head at every moment.  You can start to do instantaneous what-if 
design experiments in your head without having to have long, drawn out 
discussions or spend inordinate amounts of time actually implementing 
a design decision before you start to see its real effects.

What's Next

Over the next few months I hope to pay down enough of my credit card 
debt so that I don't feel so financially oppressed and so I have more 
than a few months of credit available if I run into anymore 
significant downtime.  That's my absolute top priority.  Second 
priority is to try to line up additional or backup consulting.  Third 
priority is to get ready for September.  Unfortunately, my agent 
project only rates fourth priority.  That doesn't mean I won't be 
working on it constantly, just that I can't work on it full-time.  But 
maybe that's what midnight engineering is supposed to be about.

It's also time for me to start getting realistic and to begin setting 
hard goals, some dates, and metrics for how to measure how I'm doing.  
It's hard to do this with a project that's not a top priority, but 
even the lowest priority project should have a plan with measurable 
milestones.  Even if I miss a milestone, at least I'll have some 
incentive to stop and evaluate whether I'm on the right path.

------

Jack Krupansky runs a one-person software business, Base Technology, 
which offers software consulting and develops object-oriented 
programming language products. You can visit his web site at 
http://www.BaseTechnology.com, send him email at 
Jack@BaseTechnology.com, or call him at 800-786-9505 (messages/pager.)