Wednesday, 6 December 2017

MongoDB Insert from Mongo Shell but not from C# API

Another annoying one that makes sense after you work it out!

I tried to run a console app which was inserting documents into a MongoDB database via a tunnel. The app had previously been used successfully on another DB that wasn't on a tunnel and didn't have any auth so I assumed it was related.

When I inserted, I got the following error:

Command insert failed: not authorized on resources to execute command

(resources is the name of the database I was using)

But when I ran an insert directly in Mongo Shell with the same creds, it was fine.

The problem? I was not specifying the database in the connection string, which means that even though the insert command was specifying the database, the connection wouldn't have authenticated the user from the resources database (I guess it would have tried to authenticate against admin or something).

Basically, instead of this:


it should have been this:


You can also use the MongoUrlBuilder, which allows you to set all the options you might need and allow it to create you the URL e.g.

var builder = new MongoUrlBuilder();
builder.Server = new MongoServerAddress("");
builder.Password = "thepassword";
builder.Username = "MongoUser";
builder.DatabaseName = "resources";
var url = builder.ToMongoUrl();
var client = new MongoClient(url);

Monday, 4 December 2017

IdentityServer 3 certificate loading woes!

TL;DR: Set Load User Profile to True on the Application Pool (advanced settings)

We have a cloud system that uses IdentityServer and loads signing certificates from pfx files in the App_Data folder - works great.

We've deployed an on-premises system based on the same code and it doesn't work. The following errors are logged:

  • ERROR Startup - Signing certificate has no private key or the private key is not accessible. Make sure the account running your application has access to the private key
Which is definitely not true. Same certs as production, same passwords, files definitely there and can be imported into the Cert store to prove it.

I ignored this initially but when we then log in over OpenID Connect, it gets to token signing and bang:

  • System.InvalidOperationException: IDX10614: AsymmetricSecurityKey.GetSignatureFormater( '' ) threw an exception.
  • SignatureAlgorithm: '', check to make sure the SignatureAlgorithm is supported.
  • Exception:'System.Security.Cryptography.CryptographicException: Invalid provider type specified.
  • If you only need to verify signatures the parameter 'willBeUseForSigning' should be false if the private key is not be available
Which is actually a pile of errors that are not actually related to the problem. I wasted a ton of time creating a different format of signing key (lots of forums say that .Net doesn't work with the Enhanced Security Provider), created some new certs directly into the local Cert store, exported them as pfx into the same location, changed the web.config to use these new keys but now the site won't load! Disable CustomErrors and now get this error:

  • System.Security.Cryptography.CryptographicException: The system cannot find the file specified.
Which is really weird since the new certs are in the same place. This is, however, a more useful error. I double-checked everything and it seemed weird until I realised that the error doesn't necessarily mean that the cert file itself is not found but it does something weird and tries to load the user profile to check certain parts of the certificate (which seems strange when loading from file but anyway...) I then found a useful blog post that says that the app pool doesn't load the user profile by default, which is why it doesn't work. IIS -> Application Pools -> Select -> Advanced Settings -> Load User Profile -> True

I enabled this and the site worked! I then reverted to the original certs and they worked too so this was simply IdentityServer covering up a random error or not providing a helpful enough error message. This also explains why it works on Azure, where the App Services system must simply enable Load User Profile by default.

Wednesday, 29 November 2017

NuGet package not visible in Manage NuGet Packages

I had published a package to a private NuGet repo and even though it was visible in one project, it was not visible in another. I looked on the server and it was definitely there.

This happened because the NuGet package was targeting .Net 4.6.1 but the project was only targeting 4.5.1 so it was simply not listed.

I updated the project to 4.6.1 and the package appeared!

Cannot install NuGet package even though it is listed

I had a package that I wanted to install listed in the NuGet packages list but after clicking install, it seemed to wait a bit and then do nothing.

HOWEVER, clicking back onto the output tab in Visual Studio (since it was unhelpfully changing to the Error List tab), I noticed that there was a constraint that didn't allow the installation. In my case, the Microsoft.Net.Compilers package was too old for the NuGet package to install.

Once I saw that, it was easy to install the NuGet package after updating the other one!

Monday, 20 November 2017

No, your web site being broken is not OK!

Those of us who write software know that we make mistakes. Developers don't consider specific scenarios, Testers miss certain tests and with the best will in the world, even the simplest applications can have bugs.


The main happy paths should work largely fine. If something goes seriously wrong for a large-scale public-facing web site, one of a number of things absolutely must happen:

  1. Ideally, the company will already know because they will get an error message emailed/displayed on a big screen/whatever
  2. If it is more subtle, maybe a user will contact the company and if this happens, it is embarrassing, so you act immediately, especially when the bug relates to a happy path that you ABSOLUTELY should have tested
  3. If it is something with a non-obvious workaround (or none at all) the Development Team make it number 1 priority and work flat out, 24/7 if required, until it is fixed. Why? Because it was a screw up that something so serious got out the door and it is a matter of quality and corporate pride that it gets fixed and quickly.
  4. The Test Manager gets a serious talking to along the lines of, if this happens again, you're fired.
  5. The Technical Team has a serious review about how this was allowed to happen and puts in place real measures to prevent a repeat the next time. This is fed back to the Management Team so that people can be accountable where they need to be - the Management Team need to ensure they are getting the whole truth, not just what someone might say to cover their own back.
What isn't OK is:

  1. Not putting any kind of banner on the web site to say that you are experiencing problems
  2. Not working with whoever found the problem to quickly work out exactly what has happened and why
  3. Telling users to delete their cookies to make it work
  4. Telling users that only some users are having the problem (as if that makes it better that it's broken for me)
  5. Not properly testing updates to consider not just the new site in a clean happy place, but what happens when a user with existing cookies and a number of different browsers comes back to a new site.
  6. Acting like a serious bug report from an end user is just business-as-usual rather than, "I'm really sorry, I'm just going to call the Software Manager to tell them" or even, "We know of a problem and the Team are still trying to find exactly what causes it".

What do we assume when we contract people to write software for us?

Many of you have been there. You need something done, you find someone, they give you a quote for the work, you agree, they do stuff then send it to you. It's just not really good enough. It's probably not terrible (but it might be) it's probably OK but there are enough things wrong with it that you can't just shake hands and pay them their money.

On the other hand, they have put in the hours, so it's probably not right just to not pay them anything but if you are going to have to mostly rewrite it - which defeats the point of getting the work done - what can you do short of legal action?

One of the most obvious things that we don't always do is write a good contract/requirements. Like a good Job Description, if you write it well, it should simply be a case of "is the person doing this?" if so, great, if not, they don't get paid.

Let us take an example. We want someone to write a plugin for Magento that handles an OAuth2 handshake for a web site. Sounds simple and it sounds like something that a Contractor would say, yeah, OK, I'll do that! But there are many things missing from such a simple requirement. One of those might be a simple question: "Have you ever written a Magento plugin before?" Why? Because although the PHP might be easy, the architecture and philosophy of Magento is not something you can simply learn from a book in a few days and I certainly don't want to pay a Contractor to watch videos and try and learn it. What if they produce something that seems to work but they did it really badly? You might not know until later.

Secondly, assuming that they have some experience that you are comfortable with, there is then a question of quality and speed. Most of the time, we are not contracting to crazy deadlines but there is still a large difference between fast and slow, especially when you are paying a day-rate of money and to make it worse, speed and quality are proportional so fast is not always good. How can you tell what their quality and speed are like before taking the plunge and committing to large amounts of money?

You can do two simple things up-front. Ask them to send you an example of some of their code from another project (or even something they have contributed to Github or whatever) does it read well? Does it look like a professional or someone who might have made something work by luck rather than skill! Secondly, set them a test - or rather the first part of the work. For example, in our example above, ask them for the basics of a plugin that doesn't do much (maybe does a browser redirect) using some hard-coded values, some basic UI - anything that should be quick and easy, the hard stuff is always in the details. You can pay them for that work if they have done OK up until now and then review what they've done and decide whether it is good and whether it matches the expectations they set. You should be honest with these people - if they don't convince you that they are producing good enough code in reasonable time, you are not going to continue to use them. Paying 2 days up-front for a project that might be 2 months long is good business! Best to lose a small amount early on and find someone else than to have to fix everything later.

Another important point is to document and communicate your expectations. If they need code to look neat, it needs to be said. Not all Developers care and if they don't know you need that, you can't complain to them when they don't deliver it. What about Unit Tests? Design sign-off? Acceptable libraries? Browser testing? If there is something complex that your project involves, can you separate that into another package and get them to prove they can do that? If not, let them do the easy stuff and pay someone else to do the hard bit.

Hopefully, you eventually find some good Contractors who you trust, whose code you know is quality, who are responsive to the work you are giving them and who are not charging crazy amounts of money in the process. This will be ongoing if your business is growing but so many of us have to use Contractors that it is a skill that your company needs to have.

Wednesday, 15 November 2017

How to interview for a Senior Developer

This is based on my experiences in the UK, trying to recruit quality people into Senior positions. My conclusion, there is a very big difference between how people view themselves and how I view the role of Senior Developer. The average salary request for a Senior Dev in the UK (outside of London) is about £50K+ ($65K+), which for many companies is a lot of money to pay out in addition to the recruiters fees, which can be anything up to about 25% of that yearly salary and all of this before you even know whether the person is any good.

I am an employer and I get nervous when I interview someone. They are usually polite and of course they can do the job that you need them to do but the simple truth is that the recruiter and the potential recruit have a virtually zero-risk opportunity to talk themselves up to convince me they can do the job. If I take the risk on them and they are not very good, I either have to let them go at 3 months, losing several thousand in recruiters fees and potentially wasting a lot of time on a person who takes more than they are bringing to the company.

If you are that person who is applying for a role at my company, what am I going to ask you?

Firstly and hopefully this shouldn't be a shock, I am going to ask you about your experience in the areas of the job description. Example: This position requires a strong interest or experience in web application security. "Tell me about your experience in web application security", "I haven't done much". "Then why are you here wasting my time just on the hope that somehow you will convince me that I should still take you on?"

We even had a guy apply for a Development Manager position and all of my questions about, "What will you need to do as a Manager that you don't currently do as a Developer?" basically caused responses along the lines of "erm...", "hmmm.." as if the person hadn't even asked himself what a Development Manager actually does.

Secondly, I will ask what it is that makes you Senior (even if you are not a Senior, I would still ask you what separates you from the crowd) and I am fed up with the number of times that the answer is basically, "It means I have more experience", "What experience do you have that a Junior doesn't?". "Ermm...."

What do you know about Dependency Injection? IoC Containers? Test-Driven-Development? Deployment? The cloud? Node js? Angular?. These are all things that I would expect a Senior Developer to understand. Not to be super-experienced: we don't all get to do these at work - but anyone with any decent interest in web design meets these subjects all across the web. Even if you don't know exactly what it is, do you not even know the basics of why an IoC container might be useful? If not, why not?

Thirdly, I will ask why you are special. So you know some stuff about .Net and you have been programming for 15 years? Top tip: I don't care about anything before the last 5 years because we don't use Web Forms, VBA or FoxPro here! We are a startup and it takes commitment, interest and passion. Don't have a blog? Why not? Your own web site? Involved in any clubs outside of work? Developer hangout events? Member of an Institution?

The simple reality is that for most of the people we have interviewed, the sum total of their CV is: I have been writing code for average companies for X years and there is nothing that demonstrates that I am anything other than a sheep who will do what I'm told but I never think of the bigger picture and my job is largely just to pay the bills.

Even though the market for Developers in the UK is massive and the supply is terrible, I will not take any person on who is asking for £50K just because they have 15 years in the business. If you want that Senior Developer job, you should love coding. You should love it so much that you can easily demonstrate how much you love it. How you owned stuff in your previous job, you were the go-to person, you built stuff, fixed stuff, upgraded it, especially when you weren't asked to do it!