Workaround: Visual Studio Debugger will not step.

January 28, 2010

Often while I’m stepping through server side code, I’ll get the error message “Unable to step. The operation could not be completed. A retry should be performed.”

Unable to step. The operation could not be completed. A retry should be performed.

Or the “Unable to step. The message filter indicated that the application is busy.” message.

Unable to step. The message filter indicated that the application is busy.

Once you’ve gotten either of these messages, neither F10 or F11 will work, you just keep getting the same message.

Finding information on this problem has been elusive, but I finally found this blog post which confirms the problem I’ve always suspected, that it’s a race condition in the VS debugger which is triggered by pressing either F10 or F5 at the same time as a javascript event is triggered in IE.

The blog post outlines 3 workarounds, none of which I care for, so I thought I’d share a little trick I’ve discovered which will usually allow you to get back on track and start debugging again.

Here’s how I get the debugger back on track:

  1. Place a break point on the next line of code
  2. Press F5
  3. When the break point is hit, F10 & F11 will work again

Warning: There have been a few times when the debugger did not stop on my breakpoint, but for the most part, it stopped 99% of the time. I may have missed a distinction in those few cases.


Time management – Don’t just work in order of importance

December 3, 2009

I think every time management book, article, & blog post I’ve ever read tells you to put all your work in order of importance and then work in order of importance. This is great, since the easiest / fastest stuff is always deceptively drawing you in, but never seems to end … and most of it really isn’t important to your goals.

However, I don’t know about you, but my most important tasks seem to be 100+ hr sub projects, and simple 5 minute job never become the most important task, even though you can knock em down in minutes!

So here’s how I attack my work now:

  1. I work on my most important task first.
  2. Then I work on the fastest & easiest tasks.
  3. Then I work on my oldest tasks.

I also pay attention to the time, and never spend more than a day on the easy stuff, so I don’t spend my whole life on it.


10 things to review after finishing your data model

November 29, 2009

When I finish modeling my database, I like to just let it sit a couple days, then spend some time just reviewing it to check for inconsistencies. The kind of inconsistencies you never notice when you’re up to your eyeballs in details, but drive you up the wall after 5 years of maintenance.

When I finished my database design a few days ago, I jotted a few things down to remember to do. But when I transcribed them into my bug tracker, where I manage all my tasks, 3 things turned into 5, then 7 and I realized if I could just add a few more, I’d have an infamous ‘Top 10′ list.

Anyway here it is. It’s not exactly comprehensive, but it’s a start.

10 things to review after finishing your data model:

  1. Naming consistencies
  2. Column default consistencies
  3. Identity / auto-number technique exists
  4. Constraint consistencies
  5. Foreign Key relationships exist
  6. Indexes on Foreign Key columns
  7. Abbreviations are consistent
  8. Abbreviations are documented
  9. Data type and size consistencies. (For example; TableA.UserName is nvarchar(N), so TableB.UserName should be nvarchar(N) as well, not nvarchar(N±X))
  10. Review all requirements again to reconfirm everything was covered

Thoughts grow to tweets, then blogs, then they just die

November 14, 2009

I’ll often want to Tweet something, but feel the need to explain further in a second Tweet. But something in that will need explaining, so it occurs to me that I really need a blog post. But then I realize I should probably post this as 2 or more separate posts to isolate ideas and keep them self contained and just link between them.

Then I realize I’ve got work to do, and drop it till I have time …
… which never comes and my thought dies having never lived.

*Even this one tweet expanded into a blog post!


… so I can worry about curly braces

November 11, 2009

I just wanted to take this Remembrance Day opportunity to thank all the Canadian soldiers, past & present, living & dead, along with those of our allies, for your dedication & sacrifice.

Thank you for fighting for our liberties so we can worry about the more important things like; litterbugs, dynamic vs. static programming languages, and the endless irrelevant debates about curly braces.

Canadian National War Memorial - Tomb of the Unknown Soldier

Canadian National War Memorial - Tomb of the Unknown Soldier


This image taken by Andrew Moor


How to enforce a foreign key constraint against multiple tables

November 9, 2009

I am building a web app with Ben Alabaster, and one of the requirements is for the user to be able to flag items for moderators. So the user can flag entity A, entity B, entity C, etc…

So I created a single flag table.

Flag Table

Flag Table

Which I then tried to tie it to the entity tables, hoping for something like

Ideal Foriegn Key Relationships

Ideal Foriegn Key Relationships

Where all the foreign key relationships were from [flag].[entity_id] to [EntityX].[id]

Then when I wanted the top 10 flags from a particular entity (B in this case), I could run a query like

select top 20 e.[name], count(*) "count"
from entityB as e
   left join flag as f
    on f.entity_id = e.id
where f.entity_type='B'
group by e.[name]
order by count(*) desc

Unfortunately, if you were to create the above table relationship, and run the following inserts

insert into EntityA( id, name) values (1, 'EntityA');
insert into EntityB( id, name) values (2, 'EntityB');
insert into EntityC( id, name) values (3, 'EntityC');
insert into EntityD( id, name) values (4, 'EntityD');

The following statement

insert into flag(entity_id, flag_reason) values(5, 'Testing without a valid FK value.');

would fail as expected, as expected, with the following error. “The INSERT statement conflicted with the FOREIGN KEY constraint “FK_flag_EntityA”. The conflict occurred in database “test”, table “dbo.EntityA”, column ‘id’.”

But

insert into flag(entity_id, flag_reason) values(1, 'Testing the FK to entity A.');

would also fail, which was undesired, with the following error: “The INSERT statement conflicted with the FOREIGN KEY constraint “FK_flag_EntityB”. The conflict occurred in database “test”, table “dbo.EntityB”, column ‘id’.” +

So, my options with regards to referential integrity are :

  1. Ditch the referential integrity, which I am vehemently opposed to. ++
  2. Create multiple flag tables, each with the exact same schema, but a different Foreign Key relationship, which just seems wrong.
  3. Managing referential integrity via triggers.

While I’m not a big fan of triggers, the ‘Managing referential integrity via triggers.’ option seems like the only tolerable one. So I added the [entity_type] column to my flag table.

Flag Table With Entity Type

Flag Table With Entity Type

Removed the relationships

No Relationships

No Relationships

And wrote the following trigger to manage the foreign key relationship.

-- =============================================
-- Description: maintain referential integrity on
-- a column which is a FK for different tables
-- =============================================
CREATE TRIGGER flag_entity_id_fk
ON flag
AFTER INSERT,UPDATE
AS
BEGIN
declare @entity_type char(1);
declare @entity_id int;
declare @cnt int;

 -- SET NOCOUNT ON added to prevent extra result sets from
 -- interfering with SELECT statements.
 SET NOCOUNT ON;

 -- get info
 select @entity_type=entity_type,
   @entity_id=entity_id,
   @cnt=0
 from inserted;
 
 -- check if records exist
 if 'A' = @entity_type
 begin
  select @cnt=count(*)
  from entityA
  where id=@entity_id;
 end
 else if 'B' = @entity_type
 begin
  select @cnt=count(*)
  from entityB
  where id=@entity_id;
 end
 else if 'C' = @entity_type
 begin
  select @cnt=count(*)
  from entityC
  where id=@entity_id;
 end
 else if 'D' = @entity_type
 begin
  select @cnt=count(*)
  from entityD
  where id=@entity_id;
 end

 -- records exist? exit
 if 0 < @cnt
 begin
  return;
 end

 -- no? error
 raiserror( 'Unable to find foriegn key match on entity type ''%s'', id ''%d''.', 16, 1, @entity_type, @entity_id);
 rollback transaction;
END

Now, when you run

insert into flag(entity_type, entity_id, flag_reason) values('B', 5, 'Testing without a valid FK value.');

The trigger doesn’t find a match in the appropriate table, rolls back the insert, and gives you a descriptive error message.

Unable to find foriegn key match on entity type ‘B’, id ‘5′.

However, a good value is accepted.

insert into flag(entity_type, entity_id, flag_reason) values('B', 2, 'Testing without a valid FK value.');

I’m still not happy with this approach, but it does seem to be the lesser of all the evils. Please let me know with a comment if there is another option I’ve overlooked. Thanks.

* Frankly I was surprised it even compiled.
+ Unless of course you were unfortunate enough to test this in a coincidental situation where all tables happened to contain the id of every test you ran.
++ Yes ‘vehemently’

EDIT (11/10/2009) : It just occurred to me that this article does not take into account what would happen if the entity tables were to delete a row which this table was pointing to. When I designed my tables this was taken into account, but since we are not planning to allow actual deletions, it was left out. However, if you were to implement this strategy, where entities could be deleted, a delete trigger would need to be created for each entity table.


How to hook up NUnit as an option in the ASP.NET MVC application wizard

November 7, 2009

Ben Alabaster and I have started a project, lets call it Project X for now (original eh? ;-) .

Anyway, we have decided to blog about its development. I will be posting something in the next few days, and Ben just published his first post “How do I hook my version of nUnit into the ASP.NET MVC template?“. Here’s a blurb:

If you’ve been looking for a way to integrate nUnit into your ASP.NET MVC 1.0 template – that is, when you create a new ASP.NET MVC application and it asks you if you’d like to create a test project, nUnit shows up in the list along with the usual Visual Studio Unit Test option.

There are a number of longwinded ways of doing things. There’s also a relatively simple way touted on the Visual Web Developer Team Blog which I’ll spare you the headache of running it and finding the same problems I did….

Ben’s post can be found on his blog endswithsaurus.com


Investigating the relationship between estimation accuracy and task size

August 19, 2009

Yesterday on StackOverflow Johannes Hansen asked

What is the acceptable upper limit of time allocated to a single development task?

I answered with

If you track your estimate/actual history, you can probably plot hours by accuracy and figure out exactly what number is appropriate for your team.

My advice sounded so good I thought I’d try it myself. So I opened bug tracker where I keep track of my probable and actual times and exported my closed bugs to Excel. I cleaned up a bit, by removing any rows with either a 0 probable or actual time, then created a chart.

Now when I conceived of this idea, I was expecting something like
Estimation Accuracy by Probable Hours To Complete - Expected

Well I wasn’t expecting the plots to be that dense, or to accelerate above 200% so fast, but let’s just say, that general look would have been pleasing to my eye.

Here’s what I got instead.
Estimation Accuracy by Probable Hours To Complete - All Bugs

Now, I’ve got to say, is NOT what I was expecting at all. You can kind of see a very dense block under 4 hours and 100%, but doesn’t tell us very much with regards to the relationship between estimation accuracy and size of the tasks. So, I then threw a Linear Regression Trendline on the chart hoping it would illuminate an ascending trend. Instead it contradicted my assumptions by declining, suggesting the larger the task, the more accurate I am … which isn’t true at all.

Maybe it’s the outliers. Maybe it’s the weird changes outside of normality causing it to look so horrible. So I sorted the data by the accuracy percentage, dropped the top and bottom 5 percent, redrew the chart and got this.
Estimation Accuracy by Probable Hours To Complete - 90th Percentile

Still obvious relationship between the estimated task size and estimation accuracy. But at least my trendline is no longer declining. By flat lining, it’s now suggesting there is no relationship between estimation accuracy and task size.

… hmmm … bugs are included in my data. I wonder if that could be having an effect? I’ve been estimating approximate times bugs will take to resolve for my manager. Most of these bugs have been estimated before even investigating the cause, so that’s not really the same as estimating a defined task. What if I remove them?

I went back to my original data dump, removed all bugs, tickets, and questions so I was left with only new tasks and changes. I again removed the bottom & top 5% and recharted.
Estimation Accuracy by Probable Hours To Complete - New Tasks Only

Well, I’ve finally got an ascending trendline suggesting my estimates are weaker as tasks get bigger, which is what we expected to happen.

Conclusion: I’m still not very happy with the scatter chart. I still believe it should look closer to my initial assumptions of what this chart should have looked like. This suggests to me that I need to take another look at my data collection if it’s going to be useful to me at all.

Feed back and constructive criticism welcome.


Two Rules for a Happy Consulting Client while Charging an Hourly Rate

July 9, 2009

Some of my projects are a fixed rate and some are hourly. If the scope of the project is small enough to accurately estimate, risks are minimal, and nailing down requirements is practical, I may propose a fixed price. Working on fixed price projects are a piece of cake, I go home, work on it, provide periodic updates via email, phone, or in person. When I finish it I install it and give them a CD with all the binaries, documentation, and source code.

However, I’ve found working on an hourly rate to be a little different than doing all the above while just counting my hours. There is a lot of distrust with hourly professionals which is only compounded with a mysterious process like software development. After all how do they know you aren’t home watching soap operas, working on your own projects, the project of another client, or even sitting on the beach? How can they tell they aren’t being over charged? The problem with trust, is even if you are honest, a false perception can still destroy the relationship.

I know this too well. Once in the late 1990s I was working from home, charging hourly, and I got a call from one of the newer partners of my biggest client (only client at that point). While talking I rebooted to clean out my system* and when the system came back up it played the 20th Century Fox theme, a configuration change I thought was very cool, and he said ‘Are you watching TV?’ I said ‘No. Why?’ … Seriously, I didn’t even making the connection. Two days later I was called in for an unscheduled meeting where my invoices were questioned, I heard a lot of “we’re not accusing your of inflating your hours…”, and the next day, feeling insulted and unappreciated, I resigned. I didn’t make the obvious connection until much later. It doesn’t matter that I was under charging both in terms of my rate and what I charged for, and would never inflate my hours! It doesn’t matter, because the perception was corrupted. It was a tragic misunderstanding since I loved working there and provided a competent, yet naively discounted service.

That’s when I came up with rule # 1
1. Work at the client site when charging hourly.

While following this rule gave my clients assurance that I was actually working, it still left them in the dark as to my effectiveness and what exactly I was doing. I soon realized that working on site isn’t enough. I need to communicate my challenges and accomplishments more effectively as well.

So I came up with rule # 2
2. Provide detailed invoices.

Until this point, my invoices had the typical, single ‘All services rendered’ line item. I changed it to include every single thing I did. I’m not kidding. Here is a sample from an invoice I wrote a couple years ago, after establishing this rule:


App F – Investigate & Fix bad XXXXX data

App A – Altered data tables to use the datetime data type for the create_dt & update_dt fields. This is needed to have a granular time for list updates. Communication /w Coder X and answering his questions. Conversation with Coder X about Stored Procedures. Discussions with Coder X about the stored procedures. Generating Business objects.

Feature X – Add Save & Save to Profile to the IFeatureX specification. Alter the Feature X specification document to include autosuggest searching ability in the IFeatureX interface. Reviewed Coder Xs progress.

Planning – Discussions with Manger Y about the purchase of App R, project status, and got permission to take the API documentation for App R off-site for review.

Website – Added new user.


Notice how it’s written for the target audience (a non-programmer, IT manager in this instance)? Notice all the detail? Notice I explained why I was changing the data types? Notice the descriptive verbiage (investigated, fixed, altered, reviewed, etc…)? Notice how even something as basic as adding a new user to the website, a 5 minute task is included? Notice I confirmed my permission to take confidential documentation off premises?

There is some terminology which might be unfamiliar to my client, but these terms were introduced to the client before hand, so they understood every thing said on that invoice when it showed up in their inbox.

All this detail may seem like overkill, but it gives the client a level of transparency into the mystical world of software development. It provides a record of activities, allowing the client to feel in control, knowing the priority decisions they made are being acted upon as agreed. It reveals a shadow of tangible evidence on an outwardly invisible service. But most of all, it gives the client comfort and I believe raises the level of trust.

I should also point out, the invoice sample above is not based on a 50-60hr work week, but from a week where I put in 14 billable hours. A 50-60hr work week would usually be 500 words or more.

… hey! Don’t freak out, you can write a detailed invoice without spending your weekend on the first draft. Here’s a brief list of things you can do to reduce the time you spend writing monster invoices:

  1. Maintain a detailed time log filled out as you do things. This takes discipline, but almost no time.
  2. Make detailed source control comments and do a report at the end of the time period.
  3. Make detailed bug tracker notes and list all the bugs/tasks/tickets you worked on during the time period. You can just list the bug ids, but I usually write something like ‘Investigated and resolved ticket # N – User receives 404 error when clicking customer link’.

The key to any of these is to write with your client as your intended audience, this way you just cut and paste into your invoice.

More than a decade after reluctantly getting into consulting, I’ve realized that Perceived value is directly related to the quantity and quality of communication with the client. An invoice can be a key communication device with your client and a powerful marketing tool if you make the effort.

*It was Windows 95 after all. At that point I was rebooting 10 times a day at least.

Copyright © John MacIntyre 2009, All rights reserved
WARNING – All source code is written to demonstrate the current concept. It may be unsafe and not exactly optimal.


A very simple Pair Programming IP Rights Agreement

June 23, 2009

Once in 1998, I sat down with my manager (the only manager I’ve ever had who could program), and we banged out some code for about 2 days. It was a very fast paced synergistic activity where one idea fed another and at the end of 2 days our initial idea morphed into something completely different and a heck of a lot better.

Well, tonight 11 years later, I’ve convinced my colleague Ben Alabaster to come over and pair program. I don’t know how it will go, I’ve got high hopes, but I am confident at the end of the night both Ben and myself will be a little better as programmers, and might have even started something worth finishing.

But two things I do know: 1) if we come up with something good, we’re both going to want to use it. And 2) if we ever get to the point of needing an agreement outlining our IP rights, it will be too late to draft one. So, Ben & I threw together some basic rules yesterday. Frankly, I’m surprised I couldn’t find any on the net already, maybe I over think this stuff more than most people, or perhaps it’s because I just didn’t look that hard.

So here’s what we agreed to:

  1. Each of us, individually, is free to use any programming concept shared, discovered, or created.
  2. Each of us, individually, is free to use anything we cocreate as part of a larger project with a significant amount of additional functionality. This can be a personal project, business project, or consulting project.
  3. Each of us must agree to release any code or binaries either as a commercial product or open source. Each of us will share any credit and/or financial profits equally.

I’d love to hear other people’s perspective and comments about this.

Copyright © John MacIntyre 2009, All rights reserved