Post-SPC14 Blues & Reviews

Head on over to the official Navantis Blog for my latest post on the SharePoint Conference 2014 – thoughts, highlights, impressions and shout-outs!

Cheers
Brian

Posted in Uncategorized | Tagged | Leave a comment

Slipstreaming issues beginning with SharePoint August 2012 CU (Part 2)

In my last post, I described the installation errors that I experienced when attempting to do a slipstreamed install of SharePoint 2010 with SP1 + August 2012 CU, and a workaround that I’d implemented in AutoSPInstaller to allow the script to proceed beyond the error. It seems however that I was a tad hasty… subsequent testing revealed that the errors were not just benign & safely ignore-able – you’d basically be left with a half-patched SP2010 installation afterwards. This was evidenced on two fronts: the version info on certain files were still showing the SP1 build, and the fact that I was able to re-run the August 2012 CU after the supposed successful slipstreamed install and could see it was actually doing stuff (instead of just quickly exiting with an “…already applied” type of message.

In discussing this problem with the usual suspects, we discovered that the August 2012 CU does indeed behave differently. Although SharePoint 2010 Service Pack 1 was nearly always listed as a prerequisite for CUs released after it, up until recently it seems it didn’t actually matter whether SP1 was installed before or after the CU. However, with the release of the August 2012 CU, this has apparently changed – the CU won’t even install on an existing farm if it fails to detect the presence of SP1.

Though we have a support case open with Microsoft about this issue, at the moment we don’t have a real solution. One viable workaround though would involve simply using a slipstreamed source containing SP1 (optionally with the June 2012 CU – the most recent one that worked for full slipstreaming), then afterwards applying the August (or October) 2012 CU. Yes, a pain in the butt… You could even go one step further and run the CU with unattended switches to ease the manual pain – probably the best approach if you’re looking for the latest & greatest SharePoint 2010 build with minimum steps & effort.

I’ll update this post as soon as I confirm any new details, either from the open support case or from the general Interwebz/Twitterz.

Update (Jan 14/2013): Trevor has posted this update based on the support case he opened with Microsoft. Let’s cross our fingers that we see a fix before (heck, or even with) SP2.

Posted in AutoSPInstaller, Bugs, Hacks & Workarounds | Tagged | Leave a comment

Slipstreaming issues beginning with SharePoint August 2012 CU (and a fix)

Ever since the release of the August 2012 SharePoint 2010 Cumulative Update (CU), I (and several others) noticed that, during the SharePoint binary file installation portion of AutoSPInstaller, it would fail with a PatchApplicationFailure error in the SharePoint installation log, if we integrated Service Pack 1 + the August 2012 CU in the SP2010 install media. Since this did not happen with any prior CU up to and including June 2012, those of us affected basically thought we’d found a bug in the CU. Famous SharePoint admin dude Todd Klindt even included it on his Bugs & Regressions page.

Feeling smug & confident that I’d been part of discovering a sneaky bug, I waited patiently for the latest October 2012 CU to be released, hoping it would provide a fix. Wellll imagine my surprise when I ran into the very SAME issue with the slipstreamed October 2012 CU media… Hang on, I thought. This couldn’t still be an issue with this latest CU. I turned to a good ol’ search of the Interwebz and found a shockingly low number of hits for the problem. Surely, if it were truly a bug then lots more folks would be experiencing it. But the top search hits actually came back to my AutoSPInstaller discussions on the issue. Hmm.

Time to try a manual (non-scripted) install of SharePoint then. Let’s see if it fails, gives a warning, or otherwise indicates a corruption of the install media. What I found was a familiar dialog box, but with a message I’d never seen before:

SP2010UpdatesNotInstalled

Note the text – “Some updates were not installed”. Well, we know that during our scripted install, some updates were indeed not installed – and this caused AutoSPInstaller to blow up & exit. However, the dialog box above simply notes it as an FYI, and allows us to proceed with the usual Config Wizard. Huh?

I did a comparison of the SharePoint installation log files – the log produced when AutoSPInstaller errored out, and the one associated with the apparently successful manual installation above. To my (repeated) surprise, both logs included the PatchApplicationFailure error. But, although this may be a new thing starting with the August 2012 CU, it apparently isn’t considered something critical enough to cause the installation to fail. Further, I noticed that both logs contained the message: “Successfully installed package: oserver” – which I take to be an indication that the setup process as a whole was a net success.

It soon became clear to me why AutoSPInstaller was bombing out. After the setup of the SharePoint binaries, the script would simply parse the log for the string “Error:” If it found at least one instance of it, it would consider the binary installation a failure and throw an error. This worked fine for every type of slipstreamed installation until the August 2012 CU. For reasons as-yet unknown, this update (and presumably all CUs going forward) does things differently to the effect that a successful installation can still actually contain errors in the log…

Luckily the fix was simple. In addition to parsing the log for the string “Error:”, I now needed to search for the string “Successfully installed package: oserver”, and modify the If statement to look for both the presence of “Error:” and the absence of “Successfully installed package: oserver” – in other words, if there was an error in the log but no message indicating overall success, then AutoSPInstaller should throw an error.

The updated AutoSPInstaller changeset that fixes this problem can be obtained as always from http://autospinstaller.codeplex.com/SourceControl/list/changesets and will eventually make its way into the default recommended AutoSPInstaller download package.

Note: At the time of writing this I realized this is seemingly not all there is to this issue. Part 2 of this post will delve deeper – stay tuned!

Posted in AutoSPInstaller, Bugs, Scripts | Tagged | Leave a comment

Using PowerShell to Automate SQL 2008 R2 SP1 Slipstreaming

Like most of you SharePoint folks, I find myself installing SQL Server quite frequently (since SharePoint has an ever-so-slight dependency on it). However I also like to have the latest & greatest service pack in my environments, and currently this means SP1 for SQL 2008 R2 (yeah I know SQL 2012 is out, but most of my clients are a bit shy with such ‘new’ releases). Since I do extreme slipstreaming of SP1 + CUs for SharePoint as described (for example) in Todd Klindt’s excellent blog post, I looked for a way to do this with my SQL Server 2008 R2 binaries.

The slipstreaming process for SQL Server is a bit convoluted, and for a guy like me, very forgettable. It is however fairly well documented in articles like this one. Therefore I sat down on evening and sought to automate the process, in order to minimize the amount of thought and manual effort required.

I wanted this script to:

  • Copy the original RTM binary files from a DVD/ISO/directory to my designated path
  • Download the SP1 packages for me (cuz I didn’t want to have to remember or hunt around for the URLs for the service packs for each respective platform (IA64, x64, x86) each time I needed to slipstream)
  • Extract each platform service pack (thus avoiding having to remember the associated command-line switches)
  • Make the required edits to each platform-specific DefaultSetup.ini
  • Place a nice little text file to serve as a label in the target path, to remind me that I’d slipstreamed this particular binary source location

Anyhow I’m happy to announce my successful first attempt at tackling the automated slipstreaming of SQL 2008 R2 + SP1 via PowerShell, and you can grab it from the TechNet Script Center Gallery, here.

Cheers
Brian

P.S. My next goal in case you’re interested is of course to automate SharePoint 2010 service pack and cumulative update slipstreaming, but due to the way MS packages their CUs this has proven challenging (apparently there are no known command-line switches for the downloaded Microsoft Self-Extractor packages!)

Posted in Scripts | Tagged | Leave a comment

What’s In Store For AutoSPInstaller v.Next

Well it’s been a little while since the last AutoSPInstaller release (and the last blog post, to be honest) but let me assure you it’s been all work and (slightly less) play! The last few months have seen a pretty intense development crunch for the automated SharePoint 2010 install/config script, and I just can’t seem to figure out when to call it quits and stop scope-creeping myself. Anyhow I think it’s time I came up for air to let everyone know what I’ve been up to.

More Of The Same Goodness, Tweaked

While there’s some notable new features (see below), one of the goals of a new release should obviously be to resolve some outstanding issues. So I managed to fit a bunch of fixes in, and here are some of them in no particular order…

First, the CreateEnterpriseSearchServiceApp function seems to have never been able to successfully have more than one query server and successfully create the query topology (for reasons that turned out to be pretty odd.) Expect a fix for that in v3.

Also, the ValidatePassphrase function will now actually check farm passphrases against all criteria (previously, the requirement for an 8-character minimum was missing).

A nice treat for me was stumbling upon http://support.microsoft.com/kb/2581903. Why? Because it explains a nagging issue I’d been having lately with the PrerequisiteInstaller.exe – namely that it would almost always crap out on KB976462 lately. Well it turns out the fix is simple – as per the article, I re-jigged the InstallPrerequisites function to install the .Net Framework prior to running PrerequisiteInstaller.exe. Done and done.

Finally and maybe worth mentioning is the small but interesting addition of a timer for the SharePoint and Office Web Apps binary installation functions. This is nice for when you’re trying to get an idea of how long each install takes (e.g. for comparing the speed of various servers & platforms).

Sure there are other tweaks and fixes here and there, but let’s get to the new stuff…

Run Once, Install Many

A much pondered (if not requested) feature of AutoSPInstaller was the ability to install and configure your entire farm from a single, central server. Well ponder no more; I’ve had a good deal of success in finally remote-enabling the SharePoint scripted process. Lots of hoops to jump through for this one, including leveraging the ever-useful PsExec.exe to, uh, remotely open the door to PowerShell remoting in the first place. I expect this feature will go through a LOT of iterations since it seems there are a ton of things that can cause remoting to go wonky, never mind trying to do a full SharePoint 2010 install over a remote session!

So far I’ve had repeated luck building 2 and 3-server farms – can’t wait to try it on larger target farms with decently-powered hardware though. Oh and one more thing – the machine on which you trigger the install doesn’t even have to be one of the farm servers…

Simultaneous Installs

Hand-in-hand with the new remote functionality is the promise of parallel installations. Some of the faithful have asked, “Hey, why can’t we have multiple binary installs going at once, since these can take a long time, especially when installing n-server SharePoint 2010 farms?” Following a suggestion that was made on the AutoSPInstaller discussion list, I’ve implemented the ability to pause after the binaries have completed installing. That way, you can safely kick off the script on as many servers as you’d like at the same time, then return to each server one at a time to press a key and configure/join the farm.

Further, if remote installs have been specified, the script will kick off simultaneous remote sessions to each server in the farm and perform the binary install portion of the script. For now, each session will wait for input (key press) before proceeding with the farm config/join, but the ultimate goal is to go fully automated and have each session somehow detect when the farm is ‘ready to be joined’.

DB Control Freaks, Rejoice

Another oft-requested piece of functionality is the ability to spread your SharePoint 2010 databases out to more than just one SQL server. This is certainly a nice-to-have for large farms where (for example) you’d like your Search databases to have dedicated hardware. Or, maybe you need to put a particular content database on an extra-secure and isolated SQL cluster instance.

The next version of AutoSPInstaller will include the ability to specify a different SQL server (and SQL alias!) for each web app, and nearly every service app you can think of. The semi-exception is Search, which does allow for a different SQL instance to be specified, but currently won’t automatically create your alias for you (though you can simply create one manually, in advance).

Even if you don’t plan on using distributed SQL servers now, but are thinking you might need to segregate your DB back-end duties in the future, you can take advantage of this new feature by creating different SQL aliases (pointing to the same SQL instance, for now). The aliases can then fairly easily be re-pointed to different SQL instances later. Cheap insurance for growing farms that aren’t quite ready to spring for all that new SQL server hardware on day one.

Choose Your Input

Last and probably least, the under-appreciated AutoSPInstallerLaunch.bat will support an arbitrarily-named input XML file passed to it as an argument. So, if you’re like me and have amassed a decent collection of AutoSPInstallerInput-<string>.XML files, you’ll appreciate the ability to tell the script exactly which XML input file you’d like to use at that particular moment (and not just one auto-detected based on server name, domain etc. – though that’s still supported.)

Aaaaannnndd a nice little feature I discovered (maybe a little late to this party) is that you can actually drag an input file onto the AutoSPInstallerLaunch.bat:

DragOntoAutoSPInstallerLaunch

That way, it gets passed to the batch file as an argument without having to type it all out in a command window – a pretty decent time-saving tip!

Coming… when?

Aha, see the note earlier in this post about scope-creep ;-) Well if I can lock things down in the coming days/weeks, I hope to check in some code that you can download and try out on your own. Something I’d consider beta I guess, although there are really two streams going on:

  • The core traditional functionality (one server at-a-time, script launched on each server manually) which is actually pretty stable and has benefitted from the fixes and features listed above
  • The new bleeding-edge remote/parallel install stuff (which can be completely bypassed by setting the appropriate input file parameters to false).

Both will of course be included in the next source code check-in, so you can decide then how lucky you feel :) You can always subscribe to updates to be notified of that imminent update!

Cheers
Brian

Posted in AutoSPInstaller, Scripts | Tagged , , | 2 Comments

Thanks!

TSPUG 2011 Community Champion Award

That is all.

Posted in Uncategorized | 1 Comment

New Add-SPProfileSyncConnection cmdlet seems to ignore NetBIOSDomainNamesEnabled property?

With the release of Service Pack 1 for SharePoint 2010 came a cmdlet called Add-SPProfileSyncConnection which finally allowed the OOB creation of a User Profile Sync connection with Powershell. Prior to this, one would either have to resort to manual creation of a User Profile Sync connection or write up some fairly eloquent custom Powershell to do the job.

As great as Add-SPProfileSyncConnection promised to be, there were some limitations discovered and not much documentation around it. Further, I think I may have found another issue with this new cmdlet – but it only applies in specific cases.

The User Profile service application has a property called NetBIOSDomainNamesEnabled, which is by default set to False. It can be set to True though if the host portion of your Active Directory domain name (e.g. corporate.local) doesn’t match the NetBIOS name of your domain (e.g. CORP). Otherwise, imported user profiles will incorrectly appear in the format corporate\brianl instead of the actual value CORP\brianl.

I’ve noticed that, on a User Profile service app with NetBIOSDomainNamesEnabled set to True, if a sync connection is created using Add-SPProfileSyncConnection and users are subsequently imported, the username format comes through incorrecty – as though NetBIOSDomainNamesEnabled was still set to False.

In the example below, you can see how, in my test domain lalancette.home, with NetBIOSDomainNamesEnabled set to True, my user profiles show up incorrectly as lalancette\username:

After_Add-SPProfileSyncConnection-Highlight

As this is unacceptable and just plain wrong, I decide to blow away the sync connection that was created by the cmdlet, and create one manually, old skool style – with identical parameters (as far as I can control). The connection gets created successfully (confession: I still get a little giddy when this happens), and I then proceed to Start Full Synchronization from Central Admin.

Monitoring user profiles within the service app page, I can actually see the number of profiles drop down to zero after that first full sync. No problem… I run it again, and presto… my user profiles return, and – most importantly – they display in the correct format!
After_ManualProfileSyncConnectionCreation

I’ve run this through several times, and can only attribute it to the connection having been created with the Add-SPProfileSyncConnection cmdlet.

Finally, I’ve confirmed this in all types of post-SP1 farms: SP1 alone, SP1 with the June 2011 CU, and even SP1 with the August 2011 CU.

Has anyone else run into this? Ping me on Twitter: @brianlala

Cheers

Posted in Bugs, Hacks & Workarounds | Tagged | 4 Comments

Powershell-based SP2010 install fails to create Version registry value?

 

As first pointed out in a discussion on my AutoSPInstaller discussion area on Codeplex, and seemingly confirmed in at least one post (that I could find so far), there appears to be an issue with one of the Powershell cmdlets that, when run with a series of other cmdlets, is the equivalent of the SharePoint Products and Technologies Configuration Wizard (AKA psconfigui.exe). The problem can be summed up as follows:

In a regular GUI-based manual installation and configuration, a registry string value called Version gets created at:

HKLM:SOFTWAREMicrosoftShared ToolsWeb Server Extensions14.0

Click to enlarge

However, if you use the SharePoint 2010 Powershell cmdlets (not sure about legacy stsadm-based configs though), the Version string doesn’t get created for some reason! This causes problems with at least one product – Data Protection Manager 2010 (see links above to relevant posts) – as well as some confusion not only as to why this would happen, but what other issues this missing registry entry might cause.

In the meantime, and to set things straight for now, I’m considering just setting the missing registry string value (currently 14.0.0.4762 for the RTM build) by adding a New-ItemProperty command in the AutoSPInstaller. However I’m left feeling that this is kinda less than ideal…

Can anyone else confirm this issue? Have you found any fixes, or are you just using a workaround like the one I’ve suggested?

Cheers
Brian

Posted in Bugs, Hacks & Workarounds | Leave a comment

(Tricks For Successfully) Creating AD Service Connection Points for SharePoint 2010

A few days ago Jie Li posted instructions on how to track SharePoint 2010 installations in an organization by using AD Service Connection points. This sounded really promising, as it could (for example) allow admins to track ‘rogue’ SharePoint installs, or at a minimum just be able to quickly determine where SharePoint’s been installed.

However, I (for one) could not get the GUID sub-container that Jie mentions in his article to be created. I initially thought the problem was that I wasn’t actually using PSConfig (or PSConfigUI), but rather PowerShell commands (through my AutoSPInstaller process). So I ran some tests using SharePoint Foundation (for speed & simplicity) via the GUI, and got some disappointing results:

  • Using the instructions exactly as outlined in the article, no service connection point got created
  • An error was logged in the Application log of the SharePoint server, stating that it had a problem with the ContainerDistinguishedName (as we were instructed to create it in the registry)

Fair enough, I thought. Guided by the error in the event log (which I should have captured for this post - my bad), I then changed the ContainerDistinguishedName value in the registry to something that actually looked like a DistinguishedName:

CN=Microsoft SharePoint Products,CN=System

[click to enlarge]

Click to enlarge

Now, I began to get different errors on subsequent PSConfig attempts – along the lines of “General access denied”. I correlated these to failure audit events on the domain controller, which indicated that, contrary to Jie’s instructions, Write permissions to the Microsoft SharePoint Products container wasn’t enough; we needed something more…

Long story short, it appears that you don’t even need Write permission to the container; it’s sufficient (but necessary) to have the ”Create serviceConnectionPoint Objects” permissions (defined on the container object only):

As you can see I’ve gone ahead and granted Authenticated Users the required privileges. Why? Because in order for this feature to be useful (and track as many SharePoint installs as possible), we need installations performed by anyone with a domain account (not just super-users) to register themselves in AD. Further, the super-granular permissions “Create serviceConnectionPoint Objects” (instead of full Write, as initially thought) should mitigate any risk of granting such broad access.

Once I performed these few steps, I ran the SharePoint Products and Technologies Configuration Wizard, and when it finished I could see the new Service Connection Point in Active Directory:

And finally, back to the question about PSConfig being required for all this to work (as opposed to Powershell cmdlets): it appears that yes, only PSConfig does the trick. There’s still something ‘missing’ in the series of Powershell cmdlets that serve to emulate the PSConfig-like activities. Not sure why but stay tuned.

Hope this helps anyone who’s run into similar issues.

Brian

Posted in Hacks & Workarounds, Instructions | Leave a comment

Automating Managed Property Mapping in MOSS 2007 / Search Server 2008

I was recently asked to customize the Advanced Search experience for a Search Server Express 2008 customer. Remember that MSSE is basically WSS 3.0 + the search components from MOSS 2007 (remember that ancient product?). Anyhow this involved enabling the selection of a number of custom fields in the ‘property restrictions’ drop-down. Part of this overall process is mapping Managed Properties to Crawled Properties within the Search Administration area of the shared services provider. I more or less assumed that this would be a GUI-only, manual process for the 20+ properties I was dealing with. However I was faced with applying the changes to at least 4 different environments – not a lot of fun, and likely prone to error by tired eyes and fingers. I wasn’t (and still am not) aware of a way to do this using STSADM.exe, and even scoured the help for STSADM once again in case I’d missed something – no luck.

As is often the case, I stumbled on a neat solution while looking for something else entirely. The folks over at mosssearch.com released a freeware utility called MOSS Search Manager a while back, and it was designed exactly for this purpose. Fantastic! Now all I needed to do was include it in a batch file, give it the names of the managed properties to map (via a text input file), and run the batch file in each environment.

Here is the full script of the batch file. Copy/paste to <something>.cmd, and don’t forget to list all of your managed properties in a file called PropNames.txt in the same directory as the script and MOSSSearchManager.exe.

@ECHO OFF
ECHO - Adding/Mapping Managed Properties...
SET URL=http://yourURL:
SET ContentSource="Content Source Name, e.g. Local Office SharePoint Server sites"
SET InputFile=%~dp0PropNames.txt
ECHO - Getting crawled properties...
FOR /F %%a in (%InputFile%) do (
MOSSSearchManager crawledprop %URL% SharePoint | find /i "ows_%%a"
IF ERRORLEVEL 1 ECHO Crawled property "ows_%%a" not found!
)
ECHO - Getting managed properties...
FOR /F %%a in (%InputFile%) do (
MOSSSearchManager managedprop %URL% SharePoint | find /i "%%a"
IF ERRORLEVEL 1 ECHO Managed property "%%a" not found! & pause
)
pause
ECHO - Mapping properties...
FOR /F %%a in (%InputFile%) do (
ECHO - Mapping property %%a:
MOSSSearchManager mapmprop %URL% %%a %%a
ECHO - Mapping property ows_%%a:
MOSSSearchManager mapmprop %URL% ows_%%a %%a
)
IF ERRORLEVEL 1 pause
ECHO - Initiating Full Crawl...
MOSSSearchManager crawl %URL% %ContentSource% startfull
timeout 10
ECHO - Checking Full Crawl Status...
MOSSSearchManager crawl %URL% %ContentSource% status
pause
EXIT

Although this script was used for Search Server in my case, the search components are shared with MOSS 2007 so the utility/script will work with both products. And yes I suppose I could/should have used Powershell, it wasn’t yet installed on the target server(s) and would have involved additional change management process – not worthwhile for the length of this particular engagement.

So I suppose the lesson is… just when you think it can’t be automated, there may be a solution out there!

Cheers

Posted in Scripts | Leave a comment