Month: October 2016

Really Wacky Exchange (ActiveSync) Error

My husband changed his Active Directory password. Routine enough – we’ve got 15k accounts at the office and require a password change every 90 days. That’s 150-200 people changing their password every day. They get themselves locked out a lot (mobile devices, cached workstation credentials, and a host of other unique places people manage to store their creds), but it’s trivial to unlock an individual user.

*Except* — after the account was unlocked, his Windows 10 mail client updated properly and was interacting with the Exchange server. Android, however, still wouldn’t accept his new password. If he typed the wrong thing, it would say invalid password. But whenever he typed the right thing, he got an error indicating the phone and tablet were unable to communicate with the server. Which was bogus — I could see the communication coming across the reverse proxy server. With 200 codes — although you can have a very successful HTTP call deliver an application error message. But it wasn’t like he couldn’t COMMUNICATE with the server. He turned sync off on the phones to avoid getting locked out again, and in the process of troubleshooting ended up deleting all of his accounts hosted on our Exchange 2013 server.

I looked through all of the event logs, Exchange logs … nothing interesting. In desperation, I enabled the individual user ActiveSync logging:

Set-CASMailbox mailNickName -ActiveSyncDebugLogging:$true

Had him attempt to add the mailbox profile again, and dropped the log myself:

Get-ActiveSyncDeviceStatistics -Mailbox mailNickName -GetMailboxLog:$true -NotificationEmailAddress mysmtp@mydomain.ccTLD

Bingo! An exception in the provisioning (Microsoft-Server-ActiveSync?Cmd=Provision) call — I see the phone information come across, the mobile device gets partially added to his account (no OS, phone number, carrier type information … but if you go into OWA and remove the mobile device, an Android device gets added). Error:

Command_WorkerThread_Exception :
— Exception start —
Exception type: System.IO.FileLoadException
Exception message: Could not load file or assembly ‘Microsoft.Exchange.Configuration.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Exception level: 0
Exception stack trace: at Microsoft.Exchange.AirSync.DeviceInformationSetting.ProcessSet(XmlNode setNode)
at Microsoft.Exchange.AirSync.DeviceInformationSetting.Execute()
at Microsoft.Exchange.AirSync.ProvisionCommand.Microsoft.Exchange.AirSync.IProvisionCommandHost.ProcessDeviceInformationSettings(XmlNode inboundDeviceInformationNode, XmlNode provisionResponseNode)
at Microsoft.Exchange.AirSync.ProvisionCommandPhaseOne.Process(XmlNode provisionResponseNode)
at Microsoft.Exchange.AirSync.ProvisionCommand.ExecuteCommand()
at Microsoft.Exchange.AirSync.Command.WorkerThread()
Inner exception follows…
Exception type: System.IO.FileLoadException
Exception message: The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Exception level: 1
Exception stack trace:
— Exception end —

Now that is an error I’ve never seen before. As a programmer, I know what it means … you’ve basically got some components that don’t match another. But … huh? He changed his password. Connected to the Exchange server directly (instead of remotely viewing logs & files) and saw Windows Update had dropped files and a reboot was pending. Which … some files replaced, others staged for replacement pending a reboot. *That* is some components not matching others. Rebooted our box, and voila … registration goes through, mailbox sync started.

I don’t know how many people allow auto-updates with a manual reboot on a production enterprise server (we manually patch and reboot during a scheduled maintenance window) where this could happen … but evidently Windows Update can get your Exchange server into a state where already configured clients are able to send and receive mail. But clients are unable to update passwords, and new clients cannot be configured.

Zoneminder Setup

I just installed ZoneMinder tonight. I don’t know if I missed a section in the documentation or something’s just missing — there’s doc for getting the build onto your box (or building your own from source). I didn’t want to install the package and all of its dependencies from their repo, so I manually installed the prereqs from the standard Fedora repositories. Installed the zoneminder rpm and switched from the ‘installation guide’ to the ‘user guide’. Which starts out on a web site. Umm … what web site? There’s definitely something missing here. Their package drops a config file in /etc/httpd/conf.d … *but* it presupposes that it is a dedicated server (which, in fairness, is the recommended configuration).

I edited the zoneminder.conf file and threw the whole thing in a VirtualHost tag, added an SSL cert for the hostname I’m using, and restarted Apache. OK, that’s better … I get *something*. Unfortunately ‘something’ is a massive MySQL error.

Some searching (rpm -ql zoneminder, then search through the files the package installed for something that looks good) yielded a config file at /etc/zm/zm.conf. Went in there, defined a database, user, and password. Created said database & user in mysql & assigned rights. Umm, stranger database error that leads me to believe the database hasn’t been initialized. Oops. LMGTFY. Found a SQL file at /usr/share/zoneminder/db/zm_create.sql and imported *that* … wasn’t sure if it was as simple as changing the zm database name in the SQL file to what I wanted or not, so I just used their default database name. Went back into MySQL and assigned the user I’d created earlier rights to the zm database and dropped *my* database. (Note, if you don’t use PHP date ‘stuff’, you may need to define the time zone in your php.ini file … I’d already set a default for other purposes).

Refresh and voila, I’ve got a system. You’ve got to click ‘start’ at the top for it to, well, start pulling in video feeds. I set up a VERY basic link to our WansView camera — ffmpg type, source is the full videostream.cgi URL, and remote method is “RTP/Unicast”. I can now see the video from our camera via the web site. WooHoo!

Now we need to get the motion detection set up & play with all of the options 🙂

Bad Deal

A friend of mine posted a graphic that basically said ten years and six trillion dollars later, we’ve got ISIS in Iraq instead of Hussein and we’ve got the Taliban in Afghanistan instead of … oh, wait, the Taliban. I understand the six trillion dollar figure looks at long term costs for veteran care *and* direct costs of the occupation. Still, the graphic got me to wondering — could we have simply purchased the country for the amount of money we will eventually spend? Iraq is 108,000,000 acres. That’s an average of 33,333$ per acre — now there are some fertile areas, some developed areas … which may well go for more than 30k per acre. But there’s a lot of desert too – not in an oil rich area – which wouldn’t go for anything like 30k an acre.

Population is something like 33,420,000 people. We could have saved near a trillion dollars ( 987,000,000,000) by giving each person in Iraq 150,000$ to do whatever we asked of them. Sure, a few would have held out … but if the alternative clearly was a foreign invasion and no 150k, I’m thinking we could have literally overthrown a government by just bribing the citizens to revolt.

Toddler Closet

I used a tension rod to create a toddler-accessible closet. I plan on raising the bar as she gets taller.

I got hangers that have clips for slacks/skirts, and each hanger holds an outfit. Her pajamas, daily clothes, and gymnastics outfits are all available. Anya *LOVES* having her clothes in her closet.

img_20161020_105423

Project Stack – Update

Halloween Circle Skirt – Completed! Circle skirts are *super* quick projects. I have a circle pattern that I re-use each time. Double-fold the bottom hem (pressing after each fold) and stitch the hem in place. Cut a 5″ wide strip of fabric a few inches longer than the inner circle circumference.  Serge it along both long sides. Placing wrong sides together, I sew the rectangular strip to the inner circle (this stitch ends up being right along the serger threads). Once the two ends meet, I serge them together. I then cut a 2″ wide strip of non-roll elastic to Anya’s waist size, overlap the ends by 1/2″ and stitch it together along the edges of the overlap and then sew an X inside the rectangle. Insert the elastic into the waistband fabric, fold the fabric over, and stitch the three layers (outer waistband fabric, circle skirt, inner waistband fabric) together. About 3/4 of the way through, I gather the already sewed part of the waistband on the elastic so the 1/4 that has not yet been sewed is straight and flat. Voila, one circle skirt.

Halloween bag – Almost done! I’m still stitching the letters onto the bag.

Halloween costume – I had to re-do the bodice (it was too wide), but should be able to finish off the back hook-and-loop closure this weekend. Need to make a hoop skirt and add some embellishment to the skirt.

 

USN Rollback

I had to recover my domain controller from the Hyper-V image backup. There’s some protection build into AD which prevents just randomly reverting a server. When you’ve got a larger domain, the built-in protection after unsupported restoration procedures serves a purpose. Pausing netlogon avoids having users log on against bad data. Disabling replication avoids propagating bad information out to the remainder of the network. The solution is simple – demote the DC, promote it again, and the DC returns to service. But when you have a single domain controller in a single domain in a single forest … well, there’s no other data around. What the recovered DC has is as good as it’s going to get (i.e. a change from 2AM is lost when I revert to my 10PM backup). And taking the entire domain down and building it overkill. You can, instead, basically tell AD to go with it. From the MS documentation:

To restore a previous version of a virtual domain controller VHD without system state data backup

  1. Using the previous VHD, start the virtual domain controller in DSRM, as described in the previous section. Do not allow the domain controller to start in normal mode. If you miss the Windows Boot Manager screen and the domain controller begins to start in normal mode, turn off the virtual machine to prevent it from completing startup. See the previous section for detailed instructions for entering DSRM.
  2. Open Registry Editor. To open Registry Editor, click Start, click Run, type regedit, and then click OK. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Yes. In Registry Editor, expand the following path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters. Look for a value named DSA Previous Restore Count. If the value is there, make a note of the setting. If the value is not there, the setting is equal to the default, which is zero. Do not add a value if you do not see one there.
  3. Right-click the Parameters key, click New, and then click DWORD (32-bit) Value.
  4. Type the new name Database restored from backup, and then press ENTER.
  5. Double-click the value that you just created to open the Edit DWORD (32-bit) Value dialog box, and then type 1 in the Value data box. The Database restored from backup entry option is available on domain controllers that are running Windows 2000 Server with Service Pack 4 (SP4), Windows Server 2003 with the updates that are included in article 875495 (http://go.microsoft.com/fwlink/?LinkId=137182) in the Microsoft Knowledge Base installed, and Windows Server 2008.
  6. Restart the domain controller in normal mode.
  7. When the domain controller restarts, open Event Viewer. To open Event Viewer, click Start, click Control Panel, double-click Administrative Tools, and then double-click Event Viewer.
  8. Expand Application and Services Logs, and then click the Directory Services log. Ensure that events appear in the details pane.
  9. Right-click the Directory Services log, and then click Find. In Find what, type 1109, and then click Find Next.
  10. You should see at least an Event ID 1109 entry. If you do not see this entry, proceed to the next step. Otherwise, double-click the entry, and then review the text confirming that the update was made to the InvocationID:

 

  • Active Directory has been restored from backup media, or has been configured to host an application partition. 
    The invocationID attribute for this directory server has been changed. 
    The highest update sequence number at the time the backup was created is <time>
    
    InvocationID attribute (old value):<Previous InvocationID value>
    InvocationID attribute (new value):<New InvocationID value>
    Update sequence number:<USN>
    
    The InvocationID is changed when a directory server is restored from backup media or is configured to host a writeable application directory partition.
    
  • Close Event Viewer.
  • Use Registry Editor to verify that the value in DSA Previous Restore Count is equal to the previous value plus one. If this is not the correct value and you cannot find an entry for Event ID 1109 in Event Viewer, verify that the domain controller’s service packs are current. You cannot try this procedure again on the same VHD. You can try again on a copy of the VHD or a different VHD that has not been started in normal mode by starting over at step 1.
  • Close Registry Editor.

 

After following the instructions from Microsoft, I still had a problem — my DC has replication turned off & netlogon comes up paused. In regedit, locate HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters and delete the “Dsa Not Writable” key (value: dword:00000004). In a command prompt, run the following:

 

repadmin /options dchostname.example.com -DISABLE_OUTBOUND_REPL
repadmin /options dchostname.example.com -DISABLE_INBOUND_REPL

Reboot the DC. When it starts, netlogon should be running and replication.