Month: March 2011
Removing server from SharePoint farm
It might sound obvious, but it is often the little things that catch out even the best of us at times.
If you’re in a situation where you need to remove a server from a SharePoint farm, then re-join it to the farm again (perhaps to resolve some sort of local corruption of the site config), make sure you manually check that all remnants of SharePoint farm membership are removed before you try adding it again. Normally SharePoint keeps a fairly tidy house and cleans up after itself well – but occassionally it leaves something behind, and this can wreak havoc on your environment if not picked up. Depending on which server role you’re dealing with in the farm, this could mean websites and/or application pools in IIS, databases, web application folders on the filesystem, or a bunch of other things.
This tripped up a colleague of mine recently – in this particular case the Security Token Service was for some reason not removed when the server was removed from the farm. When the server was re-joined to the farm, there were no errors, or issues that suggested there was a problem, but things started falling apart shortly afterwards.
How we located the issue was that the search functionality within a SharePoint site on the farm started returning those lovely generic SharePoint errors, and digging through the logs we found that old chestnut “Object reference not set to an instance of an object”.
Removing the server from the farm again, manually removing STS from IIS (including its application pool), then re-adding the server to the farm solved the problem immediately. Presumably there was some GUID under the hood that binds the STS to the farm, and as a result of the remove/join this GUID got out of sync with reality.
OCS 2007 Error 515 – “failed to execute registration stored procedure on the back-end”
Recently, after applying SQL Server 2008 SP1 to the SQL server hosting our OCS databases, the following error was observed on the main OCS server when a user tried to login via the OCS client (the user received a ‘server unavailble’ error).
Event Type: Error Event Source: OCS User Services Event Category: (1006) Event ID: 30962 Date: 2/19/2008 Time: 3:56:19 PM User: N/A Computer: Computer_name Description: Connection to back-end database succeeded, but failed to execute registration stored procedure on the back-end. This error should not occur under normal operating conditions. Contact support services. Back-end Server: Server_name Database: rtc Sql native error: 515 Connection string of: driver={SQL Server};Trusted_Connection=yes;AutoTranslate=no;server=wn4219;database=rtc; Cause: Possible issues with back-end database. Resolution: Ensure the back-end is functioning correctly.With nothing in the SQL logs to suggest a SQL error, and nothing additional in the client trace logs, I poked the haystack a couple of times to see if any needles fell out, but alas nothing did.
Came across a single post that suggested a solution – that being the following hotfix –http://support.microsoft.com/kb/949935/en-us. However while the error was identical, the symptom wasn’t. All OCS services were starting perfectly fine – to the point there were no errors logged on the OCS box at all when the services started. That said, tried the hotfix – didn’t help (no surprises there).
At this point a closer look was taken at the OCS databases for anything amiss. The only thing out of the ordinary was that within the rtcdyn database, both the RTCHSUniversalAdmins and RTCUniversalService logins were listed within the security section as being disabled (down-arrow icon on their user icon). A little more digging found that these accounts were not disabled at the server level within SSMS, and infact for the two other databases in which they have permissions, they were showing as enabled there as normal.
Immediate thought was to simply overwrite the permissions to resolve the issue, however trying to make any changes to these users against the rtcdyn database resulted in a SQL error along the lines of “User does not exist or you do not have sufficient permissions”. Given I was a sysadmin on the box, the latter seemed unlikely, so clearly there were some issues with the accounts.
In the end, the resolution here was to delete the logins from within the rtcdyn database (not from SQL entirely), then re-add the users to the rtcdyn database – making sure all database permissions were replicated on re-addd.
Note | Before being able to delete the users from the database, you need to change the ownership of a couple of schema objects (SQL will quickly tell you which ones by way of a handy error message). Make sure you change them back afterwards!
Quick OCS service restart once done and all systems back to normal.
Migrating SmarterMail 5.5 to 8.0 and IIS7
Like any good SysAdmin, I have a private hosted server on which I run my own mailsever and websites, and generally use as a bit of a sandbox for testing stuff out. Judge me if you will.
My particular server is a VPS hosted by the awesome folks at KickAssVPS (they seriously do btw), and I recently decided it was overdue for an upgrade from Server 2003 to Server 2008. No worries – just a case of migrating some websites from IIS6 to IIS7 (sadly not IIS7.5 – R2 not available at KickAss yet). What I hadn’t planned on was finding out my mailserver software was 3 major versions old – SmarterMail 5.5 to be precise – and version 8 had just dropped that week. Too good to pass that up. Shiny!! Must have!
So, now I’m migrating from Server 2003 to Server 2008, IIS6 to IIS7, .NET2 to .NET4, and SmarterMail itself from 5.5 to 8.0. I’ll admit I started to wonder if I’d just made a foolish mistake.
Migration attempts 1 and 2 were a nightmare. Mail service refused to start, webmail couldn’t connect to services, and general hair-pulling ensued. Google unfortunately didn’t have the answers – seemed nobody had tried this particular array of upgrades in one go before. I was in no-mans land. It’s lonely there.
But eventually, I figured it out. If you’re curious, here’s how…
- Make sure there is beer in the fridge.
- Install SmarterMail 8.0 on new server
- Follow the steps here – http://portal.smartertools.com/KB/a1485/set-up-smartermail-as-a-site-in-iis-70.aspx – to convert the webmail interface to IIS7.
- Stop the SmarterMail service on both servers
- Stop the SmarterMail service on both servers
- Copy the following XML files from the SmarterMail program folder on the old server to the same place on the new server: (assumes x64 OS)
- Copy the contents of the entire mailbox folder located (by default) at C:SmarterMailDomains to the same place on the new server.
- Recursively delete all copies of mailbox.cfg from the C:SmarterMailDomains folder (“del /S mailbox.cfg” will do the trick). Restart the SmarterMail service on the new server.
- This resolves a problem whereby large numbers of emails show in webmail as ‘email no longer exists on server’. The mailbox.cfg file gets rebuilt automatically when you login to webmail, so don’t panic about deleting it.
- Restart the SmarterMail service on the new server.
- Retrieve beer from fridge
- Open beer and enjoy.
Bare in mind this doesn’t take into account your need to plan your MX record cutover process. If you’re managing a mail server, I’d expect you to understand how that bit works without having to explain it.
SmarterMail 8 is a massive improvement by the way. Faster, much sexier to look at, and just generally betterer.
On The Socials
‘Best Practice’ | Enough with the nonsense
March 30, 2011
Best Practice
No Comments
gingerninjanz
I’m not sure about you, but I hear these types of things a lot in my work with clients and vendors..
“Vendor design must follow best practice”
or..
“Is this solution best practice?”
And it drive me nuts. Because if we were being really brutally honest, the answer is usually no. Best practice is not a boolean value. Its a moving target based on a combination of the things that are important to the business. Cost. Process. Performance. Manageability. Skillsets. I could go on. I tend to summarise them as ‘the real world’.Read More »
Rant