His list of things is a mixture of surprisingly sensible advice on general style (don't be repetitive, don't be pompous, be nice and thank people for their letters), content (refer to the inherited economic situation at every opportunity) and really random concerns about punctuation and grammar.
He also seems to have been influenced by Strunk and White, as his advice also includes 'if in doubt, cut it out' (which is good advice if you are writing formal documents), and 'in letters, adjectives add little, adverbs even less'. (There is a point here, although it cannot be taken too literally, as 'little', 'even' and 'less' are all adjectives or adverbs.)
According to the Independent article I linked above, his 'rules' include the following:
Use active voice and present tense.This is a well-worn rule. It's good advice in some ways, because people do sometimes use passive sentences to 'pad out' their writing, but there's so much nonsense written about it and people are so demonstrably unable to tell what's active and passive anyway (scroll down to section 3 at that link for examples), it's not actually very helpful advice.
Don't use 'impact' as a verb.Lots of people have peeves about words that were originally nouns being used as verbs. That's fine. It's illogical, because so many of our verbs were originally nouns it seems silly to pick out just one or two, but whatever. I think it's because this particular one is perceived as a 'management-speak' buzzword, which is indeed annoying.
Don't use contractions.Fair enough. Formal writing does usually avoid contractions (so 'don't' should be 'do not', for example). I used to enforce this quite strictly in student essays, but these days I let it go, as I recently noted, because I'm on a mission to discourage the lumpen, clumsy, underconfident writing style I see too much of. I don't know what kind of letters these civil servants write; if it's very formal then they should follow Gove's rule, but if they want to adopt the 'warm tone' he elsewhere encourages, I'd use contractions.
While 'best-placed' and 'high-quality' are joined with a dash, very few others are.Bit of a weird thing to say. There are well-established rules about when you use a hyphen. There are some that are a matter of preference, such as with prefixes (so some newspapers prefer to hyphenate 're-think' while others prefer 'rethink'), and these are (or should be) flexible enough to allow for violations in cases of potential ambiguity. If we turn to examples of the type Gove cites, there is a rule: 'best-placed' and 'high-quality' are spelt with a hyphen if they are used attributively (which basically means before a noun, like I used 'well-established' just now) and not if they are used predicatively (which basically means after a verb like 'to be', as in 'the rules are well established'). You would not write 'the food is very high-quality', for instance. So it's daft to pick out two random examples and incorrectly state that they always have a hyphen and incorrectly state that others don't, when you could just follow the correct way we're all already doing it. But hey - what do I know.
Don't use 'unnecessary' capitalisations.Agree. Some people like to use capitals to make words seem more important, I think. Capitalisation rules are pretty arbitrary (compare English with German, which capitalises nouns) and it has changed even since I was at school (when I was taught to capitalise seasons), but there are rules and not following them makes you look like you don't know them.
Replace 'ensure' with 'make sure'.OK. Random, but I guess it seems simpler.
Don't start a sentence with 'however'.As I recently wrote, 'however' is tricky. Sometimes people introduce a silly rule in order to rule out a genuinely incorrect usage without having to explain its complexity, but in the process rule out a lot of other correct usages. 'Don't end a sentence with a preposition' used to be one of these. I have a strong suspicion that the common ban on first person pronouns in essays is one too - if students can't write 'I', they can't write stupid waffly phrases like 'I believe that'. Likewise, banning sentence-initial 'however' would also rule out some incorrect used of 'however'. But it would not catch those I complained about in my post linked above, and it would rule out a lot of perfectly fine ones. So I think this another Strunk & White rule, who apparently allow uses like 'However much you complain, I'm not going to stop doing it' but dislike it when it's used with a comma: 'However, we were unable to change her habits'. This is silly, out-of-date advice which will lead to old-fashioned, distant writing. I'm saying nothing about whether that says anything about Gove's character.