Application architecture concerns

Sponsors interests

Know who the sponsors are. What their motivations are in bringing a software product. Is it a brave new thinking and disruption in terms of a idea that they want to realise or is it something that they want to bring in as a refinement over a legacy product that lacked architectural thinking. 

Will architecture change because of the above?  Architecture for a new idea or concept is not a real need in my opinion. A brand new product which is yet to see its customer base has other challenges like bringing a early version to the market as a Beta even to understand and refine the thought. Architecture for this would come in the way of realising this early versions. Short cuts even if they mean rapid prototyping with minimal design thoughts are good enough. When there is a demand for the product and it is steadily growing, then things like scalability, performance, availability all comes to the fore. What this means is architecture comes at a much later stage in this case.

The second category is a total product rewrite to replace a legacy product. This comes with a clear idea of existing customers and potential new customer base. This definitely require a clean architectural thinking right from the beginning. Architecture helps to avoid making mistakes of the legacy product that would have made it a burden to maintain, evolve or penetrate new markets. That normally would be the reason why sponsors would want to bring a architecture base. 

Bringing in a architecture early on structures the thinking around all the issues that floats on the surface of the legacy product over time. Also, a new architecture helps to get a leap to a different level taking advantage of the technical, technology evolutions since the legacy product. 

A third category is about re-factoring an existing product. This also require architectural thinking. But this most often may not be easy to have a cleaner structural arrangement as a total product rewrite. Re-factoring is mostly the case with legacy products as against a complete rewrite due to the cost, time pressures to bring out a better version to the customers. Often if the original product was shoddily designed, customers would bring in the pressure to have a better version as they would be bearing the brunt in terms of frequent patch upgrades. This triggers a re-factoring rather than a rewrite to allow for a quicker and better release. 

A re-factoring is always tricky than a rewrite because of the fact that it is always difficult to make sense of some one else's code. It is always rare to see a design or architecture document that is up-to-date reflecting the code which makes it even more difficult to achieve a clean partitioning. Thus re-factoring majorly is always a dream unless the original design team is remaining intact in full or part.

Application architecture concerns

Continuing the key concerns on the application architecture,


Cost of product development plays a big role

If the proposed architecture results in a lot of labour it is usually shot down no matter how good it is. This can also happen due to lack of skill thereby requiring more time to realize the architecture by the team. Budget is always constrained in organizations, especially for new ventures. Architecture will be even more constrained when developers decide to code something quick and be able to handle the short term need. If the product becomes a hit, demand picks up and all the issues come to the fore.

If it is a web application, this model works perfectly fine as the initial budget is invariably small and nothing in the sense of a scalable, reliable architecture can be done. The broad outlines can be specified in terms of the components, interfaces, functions and then the plane takes off. If the product is a hit, more funds are available and scaling can be addressed at that time. Most of the time the users are also individuals and the fee collected per them is not significant to be precise in terms of delivering a fool-proof architectural answer.

However, if it is a enterprise application paid in millions by the customer and used by millions of people of the customer such as a telco application, then short-term solutions turn into nightmares later driving the project into a death spiral and ending up having a irated customer. Architecture in such a case should be given a serious thought. In this case, the cost of having a architectural underpinning is usually insignificant compared to the size of development. Or in other words the cost of architecture should not exceed 10% of the product development cost to give a rough idea. This I believe if executed correctly should give the benefit of reduced labour resulting in lesser cost of incremental development of the product and a quicker turnaround time in releasing new features.

 It would be suicidal to go with a non-architecture approach for a large enterprise application. As the number of users increase, the size of data goes up, the rate of failures due to bugs, the predictability of performance and scaling comes to the fore without a sound architecture. Bad architecture always comes to haunt a product team at a later stage when the number of users increase. Citing cost or schedule pressures in avoiding architecture is not a sensible thing in this scenario as cost due to adhering to a architecture is always lesser when compared to the cost of an adhoc development and resulting rework.
 
A well thought out architecture results in the simplest and best way to address a problem. However, this means the problem should be correctly stated as much as possible. If this involves a cost, it is  money well spent. The hassles of spending money on a imaginary problem is far worse. Often cost overruns happen due to incorrect perception of the problem and a improper answer. For example, a requirement that says the requests will hit the application at 10 TPS (transactions per second) if in reality is at 100 TPS,  not having a Cache or a light weight framework in the architecture may cost the day for you.

There is no idealistic architecture. It evolves over time. Thus, it is important to spend money to refine the architectural thinking with prototyping. If multiple options can be tried quickly and the best technology or approach is identified, then it saves a lot of money.

Application Architecture concerns

Not all the applications need a architectural thinking. If you are building a product that has to stand the test of time or go for a long innings then you better get an architectural underpinning for that. If it means a large number of end users or it is going to run on the cloud, then it is a must to have this aspect nailed down. Most of the web scale products we see like Google or Amazon platform would not have been developed with a scalable architecture on day one.

But what I have seen is, even if you want an architecture underlying your software, and even if some one is deputed on that task from day one, the final architecture is proven only when it meets the desired scale, performance, flexibility over time. This will not happen in the initial days or months.

Architecture cannot be fixed in the initial weeks or months of a product and then onwards we can see the product following the same. It is absolutely not possible. Of course, the initial structure of the product can be laid out based on the requirements available at that time. However, as we have seen, requirements are not cast in stone. It is almost close to making the writing on water. It changes over time so drastically. There is nothing wrong about it if you leave out the 'get me this out by Monday' type of pronouncements. That's the way things are. In a way the 'agile' methods make more sense to the sponsors due to this. They do not need to wait infinitely to see the product in its full glory. Because requirements can change, architecture, design changes, implementation changes, if some one decide to wait for the first perfect release it becomes a mythical thing. So, give me value for the money I put in. Show me something that is in tune with the current stated requirement. That seem to be the mantra. In a way, the architecture also has to be moving in this path. It is true that architecture cannot be so damn fluctuating as a product evolves. The basic structure should be in place to address the key concerns. Then it is a matter of how course corrections are done to accomodate the NFRs like performance, scalability etc. That does not mean the initial architecture focuses on functionality only. Functionality should be obviously addressed. But care should be taken to ensure non-functional aspects like performance is also discussed even in initial stages.

So it becomes important to focus on the key concerns and what could they be?

Before even looking at technical or technological considerations, some of the key aspects that has to be understood as a architect is

- What kind of audience will be using this product?

In most cases, applications are used by people who may be well versed in the business domain. For example, if it is a application that automates the processes in a automobile manufacturing industry the product should be talking about their final product which are the cars and how things move around to get the processes involved in getting a car ready. The domain people who are well versed in a manual setting in manufacturing a car are the audience, apart from administrators who are a little bit more tech savvy and who may define workflows or rules or the CEO who is more interested in a summary outcome. Thus it is important to understand these people who will expect different things from the product you give out. The product should be able to meet all of these diverse view points and their skill or capability to comprehend out of the product and map it to their own needs. You cannot ask a supervisor of the plant to define rules. This has to be done by some one who is computer savvy, trained to handle the rules and processes. The CEO cannot be asked to define rules as well :-)

These are very finer details not touching any of the details of the product you make but more from the view points of its users.


- The ecosystem where the product will be deployed

Normally with tiered architectures, the deployment is going to be clients and server side. However, there could be people on the move such as salesmen or people who need to monitor things around the manufacturing facility. This may require mobility involving smart phones or tablets and others. This will impact the amount of data you can push around or other latency considerations. The product may have to be running on multiple sites across a country with local caching and ability to handle synchronize it with a centralized DB. The product may be a carrier grade one requiring geographical redundancy requiring the architecture to bring in questions early on during architecture definition such as the choice of cache or messaging technologies.

Will discuss more in the next blog..

Application architecture - a general thought flow

Slightly deviating from semantic web, I would like to talk about application architecture.

To my mind, architecture always resembles structuring the concerns. Whose concerns? Concerns of the stake holders or people who have a vision of what they want. When you build a house you have a idea of what all you need. Your wife has something else. Your kids add something more and so on. The architect has to find a balance of your concerns, create a structure on which your concerns are addressed. This means a building plan where each of your concerns are addressed. Not only that there is a certain elegance when you look at the plan in which the concerns are balanced out with a sense of aesthetics, symmetry etc. All of these are well balanced with the constraints that are there. For example, cost could be a constraint or a need to have a maintainable flooring could be a concern. Based on that the choice of colors or the ventilation and lighting, minimizing wastage, increasing utility and so on.

Balancing out the concerns, constraints, elegance on a structure is the architecture.

If it is software, the concerns are different. It could be like a sponsor saying 'I need to handle 5 million subscribers with a sub-second latency when they access a certain feature'. Or it could be like a 'highly usable interface that talks the native language of the user'. Or it could be even a schedule or cost input that says ' I need this before christmas' when you are in June. That may severely constrain the choices you have.

As you see concerns are something that a stakeholder feels a feature to be important. But further discussions may relax them or you may be able to find the range. In the example above, on discussing you may find out that the sub-second latency can be up to 3 seconds and it is still fine for the stakeholder as you may point out that the hardware cost is higher if it is kept under 1 second.

Structuring the concerns and constraints into software systems, components is a art. As there is no single right way. If you structure your application in a way it allows greater flexibility, for example changes to the data model should be done in a jiffy with configuration without any coding is something desirable. But this will mean the components involved are coded in a data agnostic way. The domain model is super-imposed on the application rather than wired within. Allowing such a flexibility may not often give the greatest performance. Thus at a architecture level, you may allow a hybrid approach wherein based on the type of transaction and the type of client accessing the system, it may be able to use the high performant route which is a more tightly coupled, specific technology stack based implementation, though it may still allow adding on a generic loosely coupled configurable implementation by the side for other types of traffic.

What if we had allowed only the high performant closed model? It may fly for the type of traffic we envisioned, yet it may increase the cost of maintenance when changes happen or some times it may not be possible to even introduce any changes. It may require a rewrite.

Thus when all concerns are brought in, the system moves in different directions. Architecture provides the space for meeting all the concerns in a way it is agreeable to the product visionaries at the same time keeps the future growth or changes in mind.

Architecture brings in malleability to the application when it is hammered with new possibilities. No rewrite or very less rewrite is needed. Only additions at the right places already provided will be needed. A good architecture cannot be there without a futuristic visualization of the requirements of an application. Architecture comes out with a lot of brainstorming rather than a single individual blowing it out. But the single individual who dons the role of the architect needs to be able to see the big picture and be able to steer the direction without getting lost in unwanted concerns. The architect should allow the different possibilities to exist but at the same time not lose track of the end goal. End goals can keep changing and hence the architect should keep the malleability to remain and not get lost during implementation to handle changes.


Similarly architecture cannot be done without understanding the technology, software and code. It is very intimately related to what happens underneath. A large number of products do not take off from the shelf. Very few meet the customer and very few among them are actually used finally. Given this, it is important for the architect to bring focus on the key concerns rather than floating on imaginary aspects. This goes against what I said above about visualizing requirements. It is important to visualize, but it is equally important to make it practical based on the ecosystem. If you have a poorly skilled team, visualizing very broad themes,  drawing the architecture and realizing them in a implementation would be a hard problem. Or if you have a sponsor who does not value the architecture and instead who is driving the short-term goals would pose a bigger problem to run a clean architecture based implementation. For architecture to bloom, the sponsor should be a technology person at heart who knows the value of a solid architecture. It is a call taken based on the commitment to make a product and take it to market and expand on it.

HP pavillion sleekbook b172TX, Windows 8 and user experience

Recently bought this laptop.
Did not really need a touch based laptop, but ended up buying it.

For reading documents or power point slides , the touch is helping more than I thought. I am able to pinch and zoom , roll over pages easily. I see the touch mainly as a mouse based screen and a little more. Gaming and paint brush kind of applications will be good to use , though I have not and may not.

The display is good but not as brilliant as my Macbook pro.

The HP laptops as I see the 'Envy' series seem to strike a middle ground between the purely tablet and purely laptop space with the touch and the key board. By having both, with less weight, and smaller screen sizes, you have a better world to manage. Especially if you are the coding kinds who keeps typing away or even the document kinds. I still don't see the soft key board on the screen as a friendly thing for typing docs or power points. One good thing about this HP laptop is it does not get heated up. The key board is sleek and similar to the Macs. I guess HP is towing the Samsung way in trying to take the Apple's position on their laptops. However, I would still any day prefer a Mac for the integrated and thoughtful UI experience.

Coming to Windows 8,  Its definitely a try I would say to get into a tile based UI. But alas, you still have the old windows desktop which keeps popping windows when you do something on a tile app. For example, when I use the tile bing and search youtube and click, it goes to the Internet explorer window to open youtube. Crazy!
Same when I use the Office 2013 power point, it seem to have a fresh new interface showing the file properties and all metadata. However when I click Save As , it opens up a window which is the old browse window to save your files. The experience is divided between the new interface and old interface. I get annoyed with the windows popping up experience after I started using Mac which has managed it more as Apps and it still has some minor pop-ups, but not in the scale of Windows.
The workspace concept by swiping your finger from left of the screen to switch across the different apps you are using (similar to alt tab) is good.
But I feel Microsoft should move away from their pop up windows and all changes to a screen should happen within the screen. I know they have a lot of smart engineers who can do this.

schema , data and semantic web

If you did not do a computer science course and specifically databases, it is unlikely you will know the term 'schema'. While many of us , even people of non-computer science background may be able to tell what a 'data' is.

What is the difference and why does it matter?

Data is all about values of some thing. For example, when some one asks your height, you may say 170 cms. The 170 cms is Data. While the tag or the name given to that value identifies what that value is in a myriad of other values such as the length of your sofa which is also 170 cms. If you need to differentiate the values or classify them as some thing meaningful, you need to have a additional tag that describes what those values stand for. So simple isn't it? 

Now I hear you telling that you knew this and you have been sending emails to your business partner who does Tshirts for you about the length and breadth of a Tshirt in cms.  Yes, you have been using it implicitly, but a computer program which is written to perform some checks, say a check that tells the width of the Tshirt cannot be more than 100 cms, will have to know to use the correct 'name' to make this comparison if it has to work across different values of the width of different Tshirts. Otherwise the program will be hardcoded to look for only 100 and it will not be a program that works for other dimensions.

Many a time in human communications, the schema is untold and left to the reader to decipher. For example, I may say to my friend, 'let us meet at the plaza to watch  'My cousin Vinny' at 7'O'clock evening'. In this, there is a lot of data such as 

• plaza
• My cousin Vinny
• 7 pm

Now, as you can see all the above are data points in this statement. However, in order to allow a machine to process this, it has to go beyond the values and be able to add tags to this to describe what this is about or the semantics of the information. Thus additional tags on the above would be

plaza - theatre

My cousin vinny - movie

7 pm - time

Now we bring the schema to these statements by additional tags. plaza is about a theatre and My cousin vinny is a movie. This kind of interpretation of the key elements in this statement helps a computer software to answer queries like 'what is the name of the movie?' or 'which theatre is being talked about' or in general even span across all statements which has theatre in it to find out things like how many web pages have 'plaza' the theatre specified and how many of them have a statement that relates plaza to My cousin vinny.

But you may wonder how on earth is it going to be possible to tag every statement, every word of what we speak and especially the world wide web. Well, to answer this, most of the web pages today have 'data' that represents government information or companies or people or others. They are all currently published from databases or even excel sheets. All of them have very rigid schema. But in the process of getting them into HTML, the schema got missed out.  

Now, all this means is to have tools that allow these additional aspects to be still maintained in the process of a HTML publishing. 

This is fine, but how about Wikipedia like pages which has lot of textual content for human consumption? 

There are efforts like DBPedia which tries to derive automatically the semantic information represented by Wiki pages. Hence, it would not be difficult to bring back the schema of the wiki pages.

This is obviously an effort and a large one. But it is happening and soon you may find the web of text look like a web of data. 

We are connected by data

Behind every social and business interaction there is data. To understand this statement, let me look at some examples.

You and your purchases


When you buy something, apart from the amount and shop's name, address and phone number etc., there is also the warranty information, maintenance contract, service centre phone numbers, if it is a EMI to be paid, then the reminders to ensure you pay properly, if warranty expires and you want to extend it, then the dates etc., if you get any free coupons, then the details of it, if you bought it as a gift for some one, then the details of that person, if you need to ship it somewhere, then the address and phone numbers, the tracking details until the goods arrived at a place...and so on.

You and your bank

In this case, sure, the bank maintains most of the details on your transactions and offers a monthly statement to you or a online statement. But when you just issue a cheque to some one, the reason why you issued the cheque or when you receive, the reason why you received the money is known only to you. The details of a credit or debit card transaction is clearly not something machine readable. For example, if I buy a laptop from Apple store, the Apple store detail is there but not the fact that it is a laptop.

Now with these simple examples, you can see the connection. What you bought and some additional details are available in the first one (with the shop) and the details of all transactions you did, not just with this shop, but with all others with other instruments (cheque) are available with the bank.

As a individual I would definitely benefit if both the above data are linked and thus makes sense to me. However, how do we make this happen? And how less painful this can be for the end user?

Semantic web is one answer to this problem. When I say semantic web, I mean standards like RDF Linked data allows to specify such linkages provided vocabulary for the above data representations are available. But beyond that every shop and every bank may have to specify using this. I feel at least the online e-commerce portals can start returning such information as a RDF/XML which can be reconciled with the banks, thus allowing a method of getting the details of all your spending automatically.

Imagine how useful this is for paying my taxes..

If the Tax department can simply accept such a format of expenses and the total of it can be shown against my income (assuming you are self-employed), then will it not save a lot of energy for everyone and of course a lot of paper and lot of tracking ?

The beauty of such linkages is that I can simply run through such a data like a breeze and look for any type of spending I did on any category, may be the shop owner can offer more discounts as the loyalty information is there up front and Income tax can reward people by offering discounts as the data is clean and available for scrutiny much more easily than ever saving lots of $$$ on being able to have more efficient tax collection mechanisms.

That's the power of linked data.

This just the surface...and if it happens it can change your and all our lives for ever.