Thursday, April 16, 2009

Science and Its Misapplication

Science is observation.

It is the systematic search for knowledge – a striving to understand the reality that we live in. It’s exciting and fun in the same way that it is exciting and fun for a young child to discover the environment that she is placed in – touching, feeling, and tasting everything. The skill with which we observe is at a much higher level, but the basic process and motivation are the same. We want to know what is going on around us.

Science is limited.

Because science is observation, the only knowledge that we can attain using it is knowledge that can be obtained through first hand observation and inference. Anything else is not science. If you hear your science professor tell you that the Earth is round and believe him, this is not science – even though she is a science professor. If your mother tells you that the stove top is hot and you believe her – this is not science.

Science is not enough.

It is sheer arrogance to think that through direct observation we can get a hold of enough knowledge to make good decisions about how to live. To re-use a previous example: if when you were a child and your mother told you the stove top was hot – the best approach is not the empirical one. Similarly, you don’t need to directly observe the effects of a severe car accident to believe those who tell you that it is important to wear a seat-belt when riding in or driving a car. Imagine how many problems we would have if before we made a life decision we had to systematically observe the exact same situation – it would be paralyzing – and ridiculous.

Science evolves.

All of these things should be fairly self-evident – I think most people would agree that science is about observation and that there is knowledge out there that isn’t observable. However, I think a lot of people miss that scientific understanding about a particular bit of knowledge evolves and is often times found to be downright wrong – to say that again with a more positive angle – scientifically obtained knowledge is constantly refining, drawing closer to the actual state of reality. Take gravity for instance – Newton was pretty sure he figured that one out – but it turns out there are situations where gravity doesn’t work like he thought. Or take tree rings – trees are typically expected to add a single ring for every year that they are living – however conditions have been observed where trees either add extra rings or don’t add any rings – contradicting previously held assumptions.

So to finish – it is important to remember that when thinking through life’s big issues: scientific observation is a phenomenal tool that we’ve developed however it is not the Inerrant Complete Answer to Everything™. Also, when thinking through a particular issue, if it is important to you to use science, it is not “more scientific” to believe someone whose profession is science as opposed to believing someone of another profession – both are faith.

Tuesday, March 31, 2009

INT v. GUID - The Battle Rages On

So a friend of mine (and probably countless other geeks) continue to debate the data type that is best used for "surrogate" primary keys in database tables....a sarcastic comment on Facebook from yours truly sparked a bloody battle in the larger war.. ok I'm being dramatic - but it did kick off a good conversation... and brought out some of the underlying issues surrounding this oh-so-important debate....the discussion follows:

---

Brian Sneddon Looks like Twitter's record ids for storing tweets is around 1.4 billion. Hope they're using bigint already because the clock is ticking...

Kyle Rowland at 3:46pm March 30
guid

Brian Sneddon at 4:43pm March 30
If you use a guid you have no way to compare the time a record was created relative to other records unless you add a timestamp field. Then if you want to be able to search off it you need to index it. Can be quite a high cost versus just using something like a bigint.

Kyle Rowland at 4:46pm March 30
Right - but you're adding multiple purpose to the key - this is kind of a hack - at bare minimum it's unclear. Justifying it in terms of processing cost is like arguing that a GUI takes too much computing cost and we should therefore use only commandline prompts (yeah, i know i know...if only)... it's the same thing though, you're trading clarity for nominal processing cost.

Brian Sneddon at 4:56pm March 30
I can see your perspective, but I don't agree that it's a hack or unclear. I think having an autoincrementing primary key is a self-evident mechanism for both establishing comparative chronology as well as ensuring a unique reference to the row. I don't think there's anything wrong with taking self-evident and consistent behavior and making logical conclusions based on it. If I store bank balance information and want to use it not only to determine a customer's balance but to also make a business decision as to what tier I consider a customer to be in if the tier is consistently determined by the range of their account balance, I wouldn't see a problem in using that account banance for multiple purposes.

Kyle Rowland at 5:52pm March 30
There's nothing *wrong* with it per se - it's just better not to tie multiple meanings together like that. The bank balance thing is a perfect example - sure you could store it like that, but what if the business rule changes...in your world that rule is now embedded as an assumption of the code that is reading the data. If the rule changes you have to change whatever code was making assumptions about the rule. If you separated them out into two distinct fields (say a balance field and a tier field) - it would be very clear what was going on, and if your business rule changed it would require minimal changes to the client code. It's not a right or wrong thing - it's a readable, maintainable or not thing..

Brian Sneddon at 6:31pm March 30
To compare apples with apples in my example, the tier never changes. If it would change then it would make more sense to store it separately. In this example a tier is fundamentally tied to a specific balance range and is immutable. Not the best real world example, I admit. However using an autoincrementing primary key still allows you to both establish uniqueness as well as provide comparative chronology in both an efficient and straightforward manner. I think the argument that it's not clear could be taken ad absurdum to claim somewhat subjectively that we should abandon any complex datatypes simply because there might be some confusion as to how they should be used. There's a balance you strike here between efficiency and simplicity, and while any point on the spectrum isn't inherently wrong there will be disagreement about the necessity of leaning in either direction. I just doubt that anyone who works with database apps would actually be confused by it.

Brian Sneddon at 6:35pm March 30
Then to get into database implementations, when inserting into an index it is cheaper to insert into the end of the index as in typical tree implementations it requires much less maintenance of the tree in the process. With guid primary keys, which is of course what this stemmed from (no pun intended), you're constantly inserting into various points in the tree and can incur a lot of maintrnance cost in the process. I just don't see a whole lot of value to using guid vs a sufficiently large integer when you need a separate primary key.

Kyle Rowland at 8:16am March 31
Of course it's subjective - this is where software is an art. I'm not saying using an int primary key is evil - just that it slightly less clear (IMHO). In general, I like to avoid doubling up on meaning - this is just one design decision - but multiply it by 1000 and depending on the collection of individual choices one of two things emerges: an elegant, modular system that is a joy to maintain or that other thing. Both work, but I know which I'd prefer to work on. So to get back to the main point - there are really two arguments that you've proposed for using int for a primary key:

1) optimized database access
2) the convenient side effect of having chronology

They're both valid... however, until you show me some evidence that #1 even makes a difference in the real world, I don't believe it should be taken in to account - and I don't mean benchmarks, I mean in the context of a real, well written app. And really in total, I'd still prefer clarity over both of these...

Kyle Rowland at 8:18am March 31
Also re: The Mythical Immutable Requirement.... I don't believe it exists. In either case here or at all.

Brian Sneddon at 10:32am March 31
Check http://www.mysqlperformanceblog.com/2007/03/13/to-uuid-or-not-to-uuid/#comment-81922 for an explanation as to why UUID can hurt performance. I know you said you don't want a benchmark, but you know I don't have access to any real-world scenerios. ;) However based on the simplicity of the benchmark at the end (i.e. bulk loading into a table) you can see that as the amount of data you need to insert rises the performance impact becomes clearly visible. This impact might not be noticable in smaller apps, but would certainly manifest as it grew.

Brian Sneddon at 10:37am March 31
of course at the same time Brian Aker's tests using UUID don't show that same performance degredation. So I guess the results aren't so clear. The great debate will live on! :)

Kyle Rowland at 11:46am March 31
yes it will :)

Saturday, March 7, 2009

Waiting to Sail

I find myself sitting in my car waiting to board a boat to Kingston - just looking around.

it is one of those times where it just seems to sink in, the insanely beautiful elegance of Creation...

The elegance of everything that the Lord Jesus placed in motion and entrusted to us is matched only by its unbelievable scale.

Looking at the water and the chaotic symmetry with which it interacts with the wind, and how the water all seems to like to stay together....that some objects stay on top of it while others are underneath.... I'm not a ph.d. In physics or biology, but all the moving pieces that are in play and doing their thing according to the Master's plan is crazy..

And that's just the water...there's also the blue sky with it's perfectly manipulated reflections and refractions of the sun's light....

And of course there are the people that are walking around the dock.....with their uncountable number of cellular (biological not telephone) operations happening constantly.... That go nearly flawlessly at all times.......

The Creator's hand is so evident and so powerful in all this, I can't help but feel small. And grateful...and in Love with the Master... Jesus.


Sent from my Verizon Wireless BlackBerry

Wednesday, November 26, 2008

The Pyramid of Software Priorities

Ok - so a pyramid is probably the most overused illustration for.....anything...but bear with me...

There are several basic priorities you need to consider when writing software (from a code perspective)... the lower priorities are foundational to the ones above them - if you don't have a priority in place, you have no business even thinking about the next level. Also, I would probably say that the higher up the pyramid you are, the more mature you are as a developer.

So what are they??


Bottom Level: "It Works" - sometimes this is all you need.

One Level Up: "It is Readable" - if your code is to be maintainable - it needs to be readable and quickly interpretable...by the next HUMAN to read it.

Two Levels Up: "It Does Not Inspire Hate" - if that next human not only is able to read your code, but doesn't hate it, you have thrown additional value into the pot -- good job.

Three Levels Up: "It Inspires" - if the next human is able to read, and actually has a positive emotional response to your code - I think you have reached the pinnacle of achievement in software development - you've added the most possible value to those that will follow you, and thus to the software product itself.

So - next time you're building something - consider things from the bottom up....and don't try to skip steps.. the more deliberate and intentional that you are about your coding, the higher up the pyramid you will find yourself.

Monday, October 20, 2008

Cause for Code

So - you get caught up in the day to day, same old routine - things get boring and stale - and sometimes you wonder why it is you do what you do. Coding takes a lot of focus - and like anything else that you have to concentrate on - it can cause you to lose your perspective. So if you're in that place right now - let me remind you why you do what you do......(and it's probably not the millions of dollars, rock-star status, and adoring-obsessive fans).....


This is from "Good To Great" by Jim Collins:

No, those who turn good into great are motivated by a deep creative urge and an inner compulsion for sheer unadulterated excellence for its own sake. Those who build and perpetuate mediocrity, in contrast, are motivated more by the fear of being left behind.


We create - we build... we pull together a myriad of different pieces of technology (most of which are far short of being mature) and piece them into working systems that not only help people accomplish - but often times, inspire and direct that accomplishment.

But take a step back with me....take a deep breath. If you notice in the passage from Collins, he discusses two motivations - which, I believe are both very prevalent in software people - the deep creative urge for excellence, and also a certain level of fear. I think everyone has a measure of both of these drivers threaded through our lives and our work - but the extent to which you can let the drive for excellence defeat fear determines the level of your work, the level of your influence on others, and the level of your own personal enjoyment of your work....(yes, you should enjoy your work - but that's a discussion for another time).

So - hopefully you know about how neat it is to nail the details of a particular system in such a way that you can just look at it and tell that it is solid and well put together. Hopefully, you are hungry for more of that. Hopefully you have that deep creative urge for excellence for its own sake....

Even though you have the hunger for excellence, fear can come into an environment in many ways - and many of us are very familiar with this. It can come from miscommunication with co-workers, being mistreated by a co-worker, or even if you mistreat a co-worker. Though, when you think about things, you can usually trace it back to its origin.

So - what is there to be done? Take responsibility for the situation - make it your mission to remove fear and feed your own hunger for excellence. If there is something that is keeping you from feeling free and empowered to do your work, identify it and deal with it - if it means doing something scary, like confronting someone, or even moreso, finding different working arrangements. Make sure that YOU are putting yourself in an ENVIRONMENT that will allow you to have your hunger for excellence. This is your responsibility - don't pass it off on to someone else.

Friday, August 29, 2008

Software Development Myth #3: My Development Velocity Is Based Solely on Me

One of my managers in the past used to say that every developer has a factor that you multiply his or her development estimates by to get a "more accurate" estimate. Ok, I believe this - some developers tend to be opptimistic in their views of how quickly they can get things done - others tend to be more pessimistic.... I even had a co-worker who thought he gave "more accurate" than other estimates...(yeah, right).... so bottom line is there's a sliding scale that you probably fall on as a developer with regards to how optimistic or pessimistic your estimations are... (I won't try to dive into the psychological reasons for how you arrived there -- anyone else want to take that post? :))

But this manager was only concerned with the developers in his environment (not surprising)... however, there is another factor to enter into the equation - and it relates to the organization - and how a particular developer relates to that organization...... so to simplify - the factor that this manager shared about is not simply to be defined by the estimating practices of the particular developer... but also how the developer works in the larger context of the organization.....

For example - at one job - I thought I had honed my estimation skills - working in a particular environment with small, packaged deliverables, and management focused on keeping hard deadlines... yeah, I could narrow things down to almost every line of code I would have to write to make something happen - not taking into account much back and forth with the folks that were requesting it........definitely easier to estimate, and it definitely pulled me from the "pessimistic" side that I tend to be on........then I worked at another company - where there was a little more back and forth - a little less rigid with regards to requirements....so my estimates probably bounced a little more toward the pessimistic side, to compensate for that.... and in yet another organization - a massive organization with many, many parties to deal with and negotiate with - I found my estimating to be on the optimistic side....and it shifted to more pessimistic, to compensate for all the challenges bringing pieces together....

I'm sure anyone who's been in the industry for any amount of time has seen similar sort of pattern... but it's something to keep in mind if you have to make a move to another organization....time moves differently in different places..... :) And you don't usually know the arate at which it moves until you have existed in it for a little while..........

Tuesday, August 19, 2008

The .NET Dilemma

So there exists a bit of a dilemma in the .NET world ... having come from the Java world recently, I think I can probably see it a little more clearly than others.... it is this:

you must use only microsoft tooling, even though it is ages behind the rest of the world

Let me give you one of a few real-world examples to kind of explain what I mean -- let's start with the "you must.." part...............I'm currently doing web apps with C#/ASP.NET - and I would have liked to use an MVC framework...HOWEVER, Microsoft's is not done yet - it's on preview 4 or something... so, the other option is to use Spring.NET or another third party MVC framework... the only thing is: I know Microsoft is working on a MVC framework - and when it's done - they'll make sure there are no competitors - that means there's already a ticking clock on the lifetime of Spring.NET and any other third party projects.... so that's the "you must" part - there's really no realistic choice other than to wait for Microsoft's stuff - or just to not worry about your application becoming *legacy* as soon as you write it.....

Ok - next part - ages behind the rest of the world -- that's right - Java (among others) has had MVC frameworks for a number of years now - they're a well understood thing - they're helpful - and darn it they make you more productive..... why is Microsoft JUST now developing something? Who knows - all I know is that it's way behind the times......

Anyway - that's the dilemma in a nutshell - the fact is - the .NET environment works just fine -- but if you want leading edge power and flexibility -- mmm.. I donno... :)