The Developer’s Pit

 

Microsoft project codename “Velocity” is now at CTP2 and due to be released in the second half of 2009. At JAJAH we have had many times to deal with scalability & performance issues, which lead us to develop our own scale-out solutions addressing both performance and high availability needs. A solution like “Velocity” could have shorten development & deployment time for us, however better late than never… There are few points about Velocity which I think are worth commenting on:

•   Velocity is an advanced distributed cache, in a way you can look at it as an ‘in memory’ database.
• , I find it interesting to learn that this solution emerged from the SQL Server group - this says  lot in my mind to the future of database architectures.
•   One of the things that .NET framework was always weak on was the enterprise software architecture, this is very much unlike J2EE that was by design including architecture and interfaces (though some of it was poorly implemented ). When talking with Microsoft people I heard that Microsoft was hoping for other software vendors to augment with additional tools, while Microsoft was trying to stay away from competing  with their own clients - this tendency seems to have changed (for the good if you ask me).
•   Velocity works well with LINQ making this a real ‘in memory database’, aside the software architecture implication, there’s a direct implication on the development process - the DB is gradually turning into a smart storage device and less as container for application logic. This allows the migration of application logic to the business layer without worrying of potential performance issues - this is a key shift in my mind that’s hard to achieve in real life today.

Some notes for the future:

•   ‘Object Oriented Database’ is very much 80’s and unfortunately did not really lift off big time - so far - however, I think it’s a gold opportunity to revisit with such a solution.
•   I’d very much like to see this integrated with .NET 4.0 parallel task framework - why have distributed cache for data components and not executable objects / tasks? Seems to me like a logical interpolation.

Some further notes:

•   The move of business logic out of the database strips the database from many of it’s assents : why don’t we see integration of database companies with storage companies?
•   The re-thinking of threads in the conceptual terms on tasks kind of reminds me “Monty Python and the Holy Grail“:

 HEAD KNIGHT:  Nee!
      Nee!
      Nee!
      Nee!
  ARTHUR:  Who are you?
  HEAD KNIGHT:  We are the Knights Who Say... Nee!
  ARTHUR:  No!  Not the Knights Who Say Nee!
  HEAD KNIGHT:  The same!
  BEDEMIR:  Who are they?
  HEAD KNIGHT:  We are the keepers of the sacred words:  Nee, Pen, and
      Nee-wom!
  RANDOM:  Nee-wom!
  ARTHUR:  Those who hear them seldom live to tell the tale!
  HEAD KNIGHT:  The Knights Who Say Nee demand a sacrifice!
  ARTHUR:  Knights of Nee, we are but simple travellers who seek the
      enchanter who lives beyond these woods.
  HEAD KNIGHT:  Nee!  Nee!  Nee!  Nee!
  ARTHUR and PARTY:  Oh, ow!
  HEAD KNIGHT:  We shall say 'nee' again to you if you do not appease us.
  ARTHUR:  Well, what is it you want?
  HEAD KNIGHT:  We want... a shrubbery!
      [dramatic chord]
  ARTHUR:  A what?
  HEAD KNIGHT:  Nee!  Nee!
  ARTHUR and PARTY:  Oh, ow!
  ARTHUR:  Please, please!  No more!  We shall find a shrubbery.
  HEAD KNIGHT:  You must return here with a shrubbery or else you will
      never pass through this wood alive!
  ARTHUR:  O Knights of Nee, you are just and fair, and we will return
      with a shrubbery.
  HEAD KNIGHT:  One that looks nice.
  ARTHUR:  Of course.
  HEAD KNIGHT:  And not too expensive.
  ARTHUR:  Yes.
  HEAD KNIGHTS:  Now... go!

  [SCENE 31]

  ARTHUR:  O, Knights of Nee, we have brought you your shrubbery.  May we
      go now?
  HEAD KNIGHT:  It is a good shrubbery.  I like the laurels particularly.
      But there is one small problem.
  ARTHUR:  What is that?
  HEAD KNIGHT:  We are now... no longer the Knights Who Say Nee.
  RANDOM:  Nee!
  HEAD KNIGHT:  Shh shh.  We are now the Knights Who Say Ecky-ecky-ecky-
      ecky-pikang-zoom-boing-mumble-mumble.
  RANDOM:  Nee!
  HEAD KNIGHT:  Therefore, we must give you a test.
  ARTHUR:  What is this test, O Knights of-- Knights Who 'Til Recently
      Said Nee?
  HEAD KNIGHT:  Firstly, you must find... another shrubbery!
      [dramatic chord]
  ARTHUR:  Not another shrubbery!
  HEAD KNIGHT:  Then, when you have found the shrubbery, you must place
      it here beside this shrubbery, only slightly higher so you get a
      two-level effect with a little path running down the middle.
  RANDOM:  A path!  A path!  Nee!
  HEAD KNIGHT:  Then, when you have found the shrubbery, you must cut
      down the mightiest tree in the forest... with... a herring!
      [dramatic chord]
  ARTHUR:  We shall do no such thing!
  HEAD KNIGHT:  Oh, please!
  ARTHUR:  Cut down a tree with a herring?  It can't be done.
  KNIGHTS:  Aaaaugh!  Aaaugh!
  HEAD KNIGHT:  Don't say that word.
  ARTHUR:  What word?
  HEAD KNIGHT:  I cannot tell, suffice to say is one of the words
      the Knights of Nee cannot hear.
  ARTHUR:  How can we not say the word if you don't tell us what it is?
  KNIGHTS:  Aaaaugh!  Aaaugh!
  ARTHUR:  What, `is'?
  HEAD KNIGHT:  No, not `is' -- we couldn't get vary far in life not
      saying `is'.
  BEDEMIR:  My liege, it's Sir Robin!
  MINSTREL (singing):  Packing it in and packing it up
                       And sneaking away and buggering up
                       And chickening out and pissing about
                       Yes, bravely he is throwing in the sponge
  ARTHUR:  Oh, Robin!
  ROBIN:  My liege!  It's good to see you!
  KNIGHTS:  Aaaaugh!
  HEAD KNIGHT:  He said the word!
  ARTHUR:  Surely you've not given up your quest for the Holy Grail?
  MINSTREL (singing):  He is sneaking away and buggering up--
  ROBIN:  Shut up!  No, no no-- far from it.
  HEAD KNIGHT:  He said the word again!
  ROBIN:  I was looking for it.
  KNIGHTS:  Aaaaugh!
  ROBIN:  Uh, here, here in this forest.
  ARTHUR:  No, it is far from--
  KNIGHTS:  Aaaaugh!
  HEAD KNIGHT:  Aaaaugh!  Stop saying the word!
  ARTHUR:  Oh, stop it!
  KNIGHTS:  Aaaaugh!
  HEAD KNIGHT:  Oh!  He said it again!
  ARTHUR:  Patsy!
  HEAD KNIGHT:  Aaugh!  I said it!  I said it!  Ooh!  I said it again!
  KNIGHTS:  Aaaaugh!

Amichay

As JAJAH CTO I get to see quite a few technologies in our space, most I suspect will not make a major impact and will end us as a nice feature at most. However, there are few technologies that reach a point where technology maturity meet with emerging customer needs. I’ll try to survey few of those in this post. I’d be glad to hear your opinion.

VIDEO CALLS

Video messaging is not new, in fact Video conferencing has been around for more than a decade and quite a few companies provide different levels of Video support as part of their offerings - but until recently it didn’t really reach wide spread as IM for example. I think it’s about to change as more and more companies will be adding video capabilities to their clients. We too at JAJAH spent quite some time ensuring our Data Platform (currently servicing Yahoo, eHarmony, eMobile and others) may pass along video. As more and more IM clients are to introduce Video capabilities the main questions are still what is the added value of Video in a discussion, and how to make money from it - yet I’m confident that in 2009 more and more main stream companies will be pushing this method of communication to the masses. It will most probably go beyond PC-to-PC communication and we’ll be seeing video calls gradually entering the mobile market later 2009. Mobile providers with growing data bandwidths and narrowing business will push this strongly and will convince us it’s the holy grail of communications - in few years from now we’ll probably not understand how we could live without it.

ULTRA LIGTH MOBILE PCs (UMPCs)

In CES ‘08 I’ve seen in Intel’s booth several bulky devices under the tag UMPCs or MIDs (Mobile Internet Devices) - none of that was really consumer ready - but you could have seen the birth of a new space of devices, a space I think will be fast growing in 2009 and effect both consumer electronics and software companies. We consume a lot of information today off the web, and the ratio of information gathering vs. information creation is continually growing - in other words - most people consume data services, but few generate information. Take a look at SHARP D4 currently sold by Willcom in Japan - it’s a really cool device if you’re a knowledge worker who needs fast Internet connection, get quite a few emails a day, need to carry around electronic documents, run presentations, get information off the web and make cheap international calls while traveling.

Take a look.

 

I also really like Fujitsu new Netbooks line - like  http://mobilitysite.com/2008/08/fujitsu-netbook-gets-official/ - though I’ve seen it in much cooler colors.

WAKE ON EVENT

Short disclaimer here - I was heavily involved in the Wake-On-VOIP project we’ve done with Intel’s Corwin Springs chipset (I might write a separate blog on how to that topic). JAJAH provided the back-end support and the client side wrappers to Intel SIP based Wake-On-VOIP chipset, while MailVision integrated with our client side allowing a softphone to be in a standby mode, receive an incoming call, wake up in less than 5 seconds and divert the call to the softphone. There was quite some press (for example ) around this project. This is actually the second time Intel is trying to introduce remote wake technologies - however -  last time their Wake-On-Lan was a total disaster. Putting JAJAH hat off for a while, there’s a broader view to wake on event technologies, allowing PCs to be in a stand-by mode yet be awaken remotely by incoming data such as emails, virus updates, OS updates, TV shows and more. As energy preservation is key these days, the ability to bring out a device out of standby mode using an external event is a key feature in many applications - specially for Mobile VoIP applications.

VOICE RECOGNITION

This will probably go beyond 2009 - but demand for high quality voice recognition is ever growing, there’s a wide range of new services just waiting for the right engine to emerge. If you know of one - drop me a note

 

Amichay

Analysts say that Mobile VoIP Market will boom around 2012 hitting more than $65B annual revenues. Many interpret this as a sign for investments in Mobile VoIP applications and services, thinking that consumer VoIP market will grow from nearly zero in 2008 to over 500M users in 2012. Every other day I read breaking news about VoIP Application for a new mobile device or platform.

I think they are all wrong. I don’t think that by 2012 Mobile VoIP market will even exist as a standalone market.

VoIP stands for Voice Over Internet Protocol - now which consumer really cares about what protocol is carrying his or her voice to the destination device? What the average consumer wants is a cheap and simple way to make calls, the protocol used is really least of the consumer concerns. Phone bill is though, but what does that have to do with Mobile VoIP? Would Internet Protocol make your phone bill shrink? I hardly think so. Phone companies need to make money so they can keep giving us a high quality service, keep their backbone, customer support, sales and other departments running requires money, and lots of it. Phone companies need to pay their employees, and these guys don’t really care if their dim comes from per minutes or per kilobyte cost. It’s not to say that I don’t think we’re over charged for making calls - we are - since most phone companies are just too greedy - but it has nothing to do with Internet Protocol per se.

So unless we switch back to direct trading economy phone companies will continue paying their employees and this money will come from us either directly (us paying phone bills), or indirectly (by someone subsidizing our calls for some sort of benefit). Mobile VoIP application providers may temporarily ‘enjoy’ lower operational costs and thus provide cheaper calling, but it’s neither convenient nor a steady service going beyond the techies community - it’s not a long term investment.

The reason I think Mobile VoIP market will never lift of, is that I don’t think it has independent existence from the mobile telephony industry. There are various providers (including JAJAH) providing all sort of mobile VoIP applications to make calls over data connection but those are not consumer ready for many reasons which I won’t count here. Mobile VoIP application can be consumer ready only if they are merged into the mobile device, either coming from the handset maker, mobile service providers or both. Hence it will never be an independent market. Mobile VoIP is a very misleading term.

What I do think will happen in the next few years is that Mobile Operators will gradually shift their core infrastructure to be NGN-like  and this is because audio is only one of the media types that will be pass over the air - it does not make sense to keep one infrastructure for voice and another for data - merging makes more sense. While this may potentially lower operational costs for mobile operators, it will not necessarily lower how much consumers pay - this has more to do with the core greediness of the operators than the core network architecture.

Best,

Amichay

Heisenberg’s Theory of Uncertainty states that at the atomic level measurements cannot be made of both the particle’s position and momentum. It does not necessarily mean that the particle does not have a specific position and momentum at the time, it’s just that those two qualitative attributes cannot be measured together. It also implies that the observer making the measurements has some effect on the outcome of the experiment, but that’s kind of beside the point here. Actually at the atomic and subatomic levels the rules of Newtonian Mechanics are no longer sufficient to explain all experimentally measured phenomena. Without going into deep discussion into Quantum Physics theories (on which I’m not an expert anyway), Quantum Physics adds the notion of Probability to the world-view of atomic level. However, if you take a physical object the rules of Newton due apply. If you take a brick for example and place it on somewhere isolated you can very well measure both it’s position and momentum and you don’t really need to know much about Probability to explain anything on the brick’s internal working. It’s quite interesting actually, on the atomic level one needs to have very deep understanding of Differential Equations and Advanced Physics, but if you take a more complex structure that includes billions of atomic particles any 6 six years old can very well describe the physics of the brick. Think about it, you need a PhD to understand behavior of a single entity, or small number of entities, but you don’t need to learn anything to to understand the behavior of a much complex structure. So somewhere, dependent on distance between elements we move from a probabilistic nature to a more subtle nature.

I worked for SAP for four years, at the time the company was around 40,000 employees. While you could very much know on daily bases what the company was doing it was very hard to measure the position and momentum of a single employee. I recall that I was amazed by the fact that the company was doing so well, when people where “somewhere” doing “something”, most of it not always related to the main focus of the company. Yet the company survives and was doing well. The parallelism between the two systems is quite striking when you think of it. Obviously if you put a person under the microscope you could ascertain the person’s position and momentum, but to clearly understand the person’s role in the company, the inter-relations with peers and to be able to predict what the person is going to do next is nearly impossible, you’ll probably need a PhD is psychology and your answer is likely to be probabilistic. It is much simpler though to analyze the entire complex, even though it’s made of 40,000 employee, most of which you can hardly understand.

The first Israeli Venture Capital firm emerged somewhere in the 80’s and it was mostly government backed. It all started upon realizing the potential of turning academic ideas into business solutions as well as harvesting the improvisation mentality of the Israeli society. This pulled more and more money into the growing Israeli Hi-Tech industry which today is one of the major drivers of the Israeli economy (if not the first). The start-up industry flourished in the 90’s till today, but there are warning signs for some time now which I think are overlooked.

There are more than a few warning signs, in my mind, which I think that unless attend to will have big impact on this industry in two to ten years time from now. My observations are based on comparison to other history of other Israeli promising industries, looking at the well from which this industry sprang from, missing evolution of the hi-tech industry, last but not least, observing the global world around us.

My first observation is historical in nature; Israel is 60 years old, virtually an infant in nation-years (kind of the opposite, I hope, to dog-years) but for me the first warning sign should be other industries which flourished here for two or three decades and are basically extinct now like textile and agriculture. They too where the forerunners of the Israeli economy, they too were based on local ingenuity and dedication and they are both dead in the water. We live today and we think we’re very different from our forefathers, but as much as my generation try to distinguish ourselves from our resent history, it will come back to hunt us - unless we have clear understanding of the current situation and the potential of what our industry may turn into. Don’t forget that the fact that we currently make a good living does not by itself guaranty it’s future.

My second observation goes to the source (in more than one way actually), the Israeli Academy from which previously many company sprang is slowly, but surly dying. I will not go into all the reasons for that, mostly because I don’t really know, but fact of mater is that faculties are being closed, curriculums are being amended and colleges now exists where it’s much easier than  before to acquire an academic degree. 10 - 15 years ago there were a handful of universities that could grant Bs.c. in Computer Sciences, it was hard to get in, and it was even harder to stay in. There was very strong emphasis on mathematics and the science behind Computer Science. It seems that in the last decade or so, studies and students shifted from Computer Sciences to Software Engineering, this change has been conscience and did not happen by chance.  It was done because industry leaders were thinking we need more people punching keyboards and people drawing on the board. When I look at people around me today, I see less and less science and more and more engineering. I think that’s wrong. Israel can not compete in quantity, even if we flood the industry with software engineers, all the colleges and universities in Israel will always create less the amount of engineers as couple of equivalent institutes in China or India. We’re just too small. I have no objection to having schools train software developers, obviously it’s important and needed, but that does not mean it has to come from the academy. It should have been kept separate, and there should be greater focus on the academic side which generates the future of the hi-tech industry.

My third observation relates to the evolution of the hi-tech industry, and I think that the best explanation to that is the fact that when people talk about the hi-tech industry the actually reference to the hi-tech start-up industry. Assuming there were great start-up companies 10, 15 years ago, there should have been great, mature companies in the Israeli hi-tech industry by now. But there aren’t, they were all acquired, or went away and only a handful turned into mature Israeli companies. I suppose that has a lot to do with the short-sighted-ness of the Israeli mentality which is base on political and security reasons. You never know what’s going to happen next when you live in Israel, so you become more focused on the near future than the long run. People just want to cash out and live their lives, rather than  build a work place for the next 10 - 20 years. But as many of us know, when things don’t move forward, they move backwards: when the industry does not ascend, it will eventually descend.

My fourth and last observation relates to the world around us, Chinese outsourcing capabilities are growing stronger by the day. Not only that, if Israel was the second or third place in the world in quantity of start-up companies, these places are now occupied by China and India; I suspect that other places in Asia (and South America) are soon to follow. It is not cost effective to hold engineering in Israel when prices are ten times lower in China and productivity is higher. You get things done quicker and cheaper in China and their quality of project management constantly improves.  

I don’t think we’re doomed yet, and there’s still an opportunity to make things right - if we plan them correctly. Our area is called ‘Middle East’, but we’re not in the middle of any east. We’re actually more of Middle West than East. Many battles were fought over this place and a lot of blood was shed because we are the bridge between East and West. We are more dedicated to hard work than most of the Western world, and we’re more free thinkers than most of the eastern world.  We should try and leverage. Here’s what I think we should do:

1. Harden the criteria for receiving academic degrees and separate between Engineering and Science. Create enough incentives for people to study and theorize. Identify the scientific segments where further research is required and make sure universities are well funded to research in those areas such as bio-technology, nano-technology and micro electronics, advanced energy sources etc. Make sure the well does not go dry.

2. Create engineering levels and certificates backed by government standards, create school and criteria’s for engineers to grow and improve. Not everybody will turn to academic life if there’s enough place in the engineering world. Make sure quality if maintained at the highest possible levels.

3. Create incentives for building long lasting companies verses quick exists. Change the Taxation system as well as government based backing to support the change. Create funds that back companies long term and turn the funds into grants if the company becomes solid and mature. Educate people and create the proper atmosphere where people understand they gain more if they plan their moves longer term, than short term. Make sure the hi-tech industry ascends.

4. Create more relationships between Asian and Israeli companies. Create direct flight routes between Israel and Asian capitals. Educate Israeli on cultural differences, bring more of the Asia culture to Israel. Create joint ventures and trading routes. If  we can’t beat them, we should join them.

Amichay

(This blog post was inspired from a discussion I had with Moti Gust earlier this week, Moti and I have been working jointly on various development projects since early 90’s)

 

Model-View-Controller (MVC) is a corner stone Design Pattern that in recent years received more and more acceptance. MVC is a “solution” to various problems raised when presentation code, business logic code and data access code are mixed together. MVC separates a UI driven application into three components: View - the presentation code (which renders the contents of the model), Controller - translates between user actions (received through the View) and actions performed by the model and Model - representing the enterprise data / business rules that communicate with the data model. The following image (taken from here) summarizes the concepts in this design pattern:

 

 

There are some frameworks that endorse MVC as the proper way to build UI applications, such as (part of the Castle Project)  and recently Microsoft has been looking into integrating MVC into the .NET Framework 3.5, lots of info on that around the web, one such link is this.

MVC is a great concept, it provides away to separate between pure UI code, UI logic and application logic. The separation allows the UI developer to be de-coupled from the application logic, all he needs is a good, robust API and to separate between the graphical / rendering part of his presentation code (the View) and the link to the business logic (the Controller). It even allows another developer, who did not originally wrote the rendering part to tie between the model and the presentation. It gives a lot of flexibility. But Dragons be here - is that really the solution we’re looking for?

Before going into the real-life example which flared this whole discussion, I think it’s worth a while to pause and think about Design Patterns. There are many ways to categorize and segment Design Patterns, I won’t get into all that, but if you look the MVC, Publish-Subscribe Design Pattern and Patterns like Singleton something very evident pops up : Singleton is implementation driven, it’s practicable (while some would argue that it’s overly used incorrectly [which I agree to, but we'll leave that for a later post, maybe]), its a technical way to write you code, it’s a tactical solution. While MVC (and others) are different, they are CONCEPTS, they are a way to think of information driven SYSTEM, it’s a strategical solution. But what MonoRail and Microsoft are doing is to endorse  MVC as a framework, rather than system’s architecture solution, and in my mind this falls short. If your development group is well disciplined and the design is good, you don’t really need a framework to uphold the rules - there are many other ways to address that. What is missing in my mind (and I’ll try to prove it with a real-time example) are system architecture solutions that address real-life application development & deployment cycles that address mid to large size projects with MVC. Instead we get some development frameworks suitable to a small group of developers working on smaller scale projects, at time it seems more of a university type of solution than issues we tackle in real life.

So after all that, let’s get to the problem at hand. JAJAH is a medium to large size software project with a wide variety of  technologies, we’re also a global company with Marketing in the US and R&D, Operations and other departments in Israel. One of our hurdles is that content changes should be close to the Marketing team, while today every request has to flow to the R&D team and be part of the release cycle. This causes extra work (tidious, not-so-interesting work for our killer developers) on the R&D team, and does not provide quick enough response to the Marketing team. Hence we need some Content Management System where we can have a not-so-expensive HTML guy next to the marketing team making content changes. This is practically MVC. However, we need different release cycles, the View should be developed and deployed in a different way than the rest of the system. It’s not just separating everything to different projects, DLLs etc - it’s a whole different system. We need to QA the content, having it in a staging environment and have proper roll-out procedures. We also want to limit the changes that the HTML guy can make in the system, we don’t want to give the guy full control over all rendering aspects, just certain parts. We used to play around with Offermatica (recently acquired by Omniture), but while it’s a really cool tool, it’s mostly focused around A/B tests and is not suitable as a heavy duty content management system.

MVC is cool, and it makes a lot of sense, but what we actually want is M-V-C, totally separating the components for that a framework is not enough. For that MonoRails is just not good enough since it keep everything together. What we need is some type of a Content Management System, coupled with a release cycle & deployment management, and same applies to the Model. The controller is a different story, at times we would like to couple it to the Model and times to the View release cycle.

My point in all this is that Design Patterns should not be regarded as framework solutions pre-say, but rather as design concepts being applied to the high level system architecture and not necessarily at the development level.

Amichay

On January 2008 I wrote a blog entry about Google’s Android and Social Network API’s under the title ‘Developer is King’ . Recently a friend directed me to Google’s new initiative : . Google is actually not the first to provide and host a network based solution for developers, a well known solution is . But while Amazon provides Web Services for storage, payment and others, Google actually took this a step forward - Google is taking this way beyond Web Services and provide a full blown Application Engine running on top Google’s monstrous infrastructure.

I think this is quite ingenious on Google’s part, they provide a fully hosted application environment, allowing developers to enjoy Google’s infrastructure for storage, load balancing and scale, authentication and various other API’s. Google’s MapReduce and GFS probably serve as an infrastructure together with their monitoring, and hosting technologies.

Each application is running in it’s own secured Sandbox allowing distribution between multiple servers and distributed web requests. Currently only Python programming language is supported, but assuming this takes off more are likely to be supported.

This move on Google’s part should attract application developers to use Google technologies as well as create an eco-system around Google web technologies. This is a bold move that will probably take a while to mature but if you’re a web application developer this is certainly something to look at.

Thinking back, it was Larry Ellison, the legendary CEO of Oracle, who coined the term Network Computer (NC)  in the mid 90’s, while the initiative itself did not really take off it created a lot of buzz that later lead into ‘thin clients’. If Google will play this right, thin applications will emerge : applications that take into consideration their distributed and scalable nature, without worrying about the complicated setup, storage and hosting environments.

While it’s hard for me to know how strategic it is for Google internally, and how they intend to push this, if at all, I think this could make a difference in the constantly evolving  computation world. I can only bow to what Google as a software giant is trying to push. If I were Microsoft I would do some serious thinking, mostly since .NET - Microsoft’s leading environment is severally lacking an application engine, but I will leave that for a later post…

Amichay

Kylie Minogue song (not that I’m such a great fan), third paragraph writes:

"you know it’s all in your head

You better put that business to bed

By your fair hands of design you met with

The monster in your mind"

when it comes to iPhone - this is so true! not that I think that Kylie is such a techno-prophet …

Apple did it again, and it’s all for the sake of the short sighed stock price and the word on the (Wall) Street. Though the writing is already on the Wall - trying to squeeze much of a platform for the sake of the product revenues is not a long-term-winning-strategy. Apple had a taste of that during the Mac-PC wars, and they lost even though their ‘fair hands of the design’ where all over the place. Google has already figured it out by targeting the echo-system, Apple however is taking a different approach. iPhone Bluetooth is blocked beyond using the headset, iPhone is locked to certain US carriers, iPhone SDK is limited.

Basically Apple is trying not to cannibalize their short term earnings from the iPhone and try to squeeze as much as possible from the product / buzz, but this is a short sighted vision. I would have liked Apple to take the lead not by having copycats grab concepts and idea from their iPhone product, but provide a winning open platform with rich API’s to really change the mobile world.

There are over billion mobile devices around the globe - aside the fashion statement iPhone’s target of 10M devices is negligible.

Apple please don’t do it again!

Amichay

 

Took me a lot of time to realize that nobody really understands networks, many say they do, many more think they do - but very few actually know what they are talking about. Specially when it gets to setting up massive data centers. At the beginning of March 2008 we plan at JAJAH to turn the lights on our new data center in New-York City. In case you want to visit us you can probably find our IT & DBA teams here around the beginning on March, but don’t piss them off - they’ve been working day and night for the past couple of months so they might be a little cranky by the time you get there.

This post is greatly influenced by our head of engineering Alon and his quest for the flawless data center.  Alon is not alone in this quest (hhm…) , this undertaking is joined by our system administrators, storage and other experts all working to get the perfect, scalable and robust solution to serve our growing user’s community as well as providing a sound base for JAJAH business growth. But before we pin the medals let me share with you one bizarre experience that led me to the belief that network is more of an ART than Science. 

Alon setup site-to-site VPN between our existing data-centers and the new NY data-center. However, when our DBA tried to transfer data from a remote database to our new monster DB machine in NY, the connection would simply hung. Alon and his team looked everywhere, checked the ISP connection, Firewall cluster, routers, BGP setup, MTU, blades center set-up, DB cluster, OS, installed and re-installed every piece of hardware - you name it - but nothing worked and time was running out. Our DBA started to get agitated (big guys - you don’t want to upset them), we had to look for a work around. We found the most bizarre workaround ever - TCP OFFLOADING.

TCP Offloading is a great feature which moves TCP stack processing from the main CPU to the network interface. Works great and improves performance, but sometimes causes problems. Just by chance we discovered that if you disable TCP Offloading and move the network processing back to the main processor things start to work well for us. Dotan, our head DBA was smiling again. Yet another day in the office.

Although we could not prove the performance lose by disabling TCP Offloading we knew that we paid for the feature, might as well get the damn thing to work. So while our DBA team was back on track we started to investigate deeper. We found two things which you don’t always read about in the school text books, but you run into in daily life.

First some background. there are two important mechanisms that play a significant role when two sites have to be connected over the web (specially when firewalls are involved): MSS - the TCP maximal packet size between two networks, and MTU size. MSS is specially important when trying not to fragment packets (for obvious reasons packet segmentation is a problem when it comes to encrypted networks). The other important mechanism is PMTUD which stands for Path MTU Discovery - an automated mechanism for MTU size negotiation when sending traffic around the web. A nice article from CISCO that explain about MSS negotiation and path MTU discovery  can be found here. 

Few things you have to be aware of when setting the MSS & PMTU:

• MSS - MSS is negotiated between the two end-points, but it is fixed in nature - the smaller of the MSS will be used by both sides.
• While MSS is set between the two ends point, PMTUD is set according to the route between the two end points.
• MTU is automatically selected using ICMP. Namely, one side sends packets in certain MTU and the other side returns notification using ICMP protocol. However, ICMP is also used with the common ‘ping’ utility (ping is actually sending ICMP and waiting for the ICMP echo to return). Because of potential ‘denial of service’ attacks many security officers block ICMP messages (Alon is also our security officer, the guy would block HTTP if we’re not looking), and this many hamper PMTUD. You need to make sure your firewall rules are set correctly to allow the relevant ICMP messages to be sent between the two VPN sites. Your access list should look something like:

• For PMTUD to work you need to make sure the DF (’dont fragment‘) bit is set in the routers.

Once Alon got the MSS correctly configured, and proper rules where applied to allow PMTUD -  we could enable again the TCP Offloading. Go figure.

Amichay

p.s.

Ori, Alon, Nir, Ilya, Eran, Dotan, Dani - this one is for you…

I write this blog entry some 10688m above a place called Yakaterinburg , flying back from Tokyo. This is my third visit to Japan in the past few months, working with our Japanese partners. A lot has been said and written about technology in Asia, and particularly in Japan. "Everyone" knows that Japanese like gadgets, but in my short visits I’ve gained some more insights into Mobile technology in Japan that makes me admire the Japanese way of thinking. A quick comment before I start, one thing which fascinated me in Japan, and I’m quite aware that this could be just my personal conditioned experiences, is that influences of tradition are more "felt" in Japan than other places I visited. While Japan is clearly far off from the Samurai days, I could still sense some of the culture and history in contemporarily Japan. Unlike places where the cultural roots seems to have faded. 

Back to technology, here are some things I picked up:

Mobile Devices are bulkier than the small, round edges devices I’m used to (I am a big fan of Nokia) : Devices in Japan tend to be big, mostly because of screen size (see below). That took me by surprise, I always thought that Japanese customers prefer small devices.

• Mobile TV - Japanese devices have the capability to receive TV broadcasts. To do that, relevant devices are equipped with a special antenna. People actually site down in trains and watch TV broadcasts.  Quality is very good. Apparently Location Based Services are also underway meaning that users receive certain local broadcasts when located in certain areas such as a community TV and others (there are some other interesting applications, but let’s leave it at that).
• There’s an interesting feature to Mobile TV - and that’s connectivity to web based information. When I was in CES earlier this year, I saw all these science fiction home TVs where one day you might be able to browse the web while watching TV (side note: why do all these demo use cases always converge to watching "Sex and the City?"). While it may still be science fiction on home usage in the US, it’s a reality in Mobile TV in Japan. That’s really cool, you can see special links under the Mobile TV screen or relevant web based tickers.
• I’m not sure why people would want to watch TV on their mobiles, on the other hand  I hardly watch TV at all (just too many things to do in real life), so I’m probably not the target customer. But fact is that this technology is commercially available in Japan makes them in my mind way more advanced than Western mobile users. I also think that Mobile Advertisement will likely to pick this up. Current Ad solutions I saw are not targeted, there’s an interesting market opportunity there in my mind.
• Mobile Data connections in Japan are far more advanced than we be found in Europe and certainly the US. HSDPA of 7.2Mb is nearly five times faster than in my home country (it’s also different frequency), not to mention hardly any packet drops (and I checked…). I also found it interesting that WiFi networks are not as common as they are in the West, seems as if people rely much more on mobile data networks than wireless devices. This really caught me by surprise.
• SMS / Text Messaging is uncommon in Japan, I saw large portion of the people in the Tokyo underground fiddle around with the mobile devices either playing, or writing messages, at first I was sure they are SMSing, but they are not - they are sending and receiving emails. While push-email in the West is mostly used by business people, in Japan everybody send and receive emails to their mobile phones. Always wired society.
• While Japan is governed by huge corporations, I found the underlying VoIP infrastructure already deployed by Telcos quite advanced as well as VoIP capabilities I did not see anywhere else. Since dealing with large, heavy corporations I expected them to be some what laid-back when it comes to VoIP, I was wrong. But for obvious business confidentiality issues I can’t elaborate.

If you do get a chance - try visiting Japan. It’s not all about technology, the people are very welcoming, food is great, culture is interesting, service is superb and there are some great hiking places in the mountains (wish I could do more of that).

Till next time

Amichay