Summit 7 Team Blogs

License usage reporting in Office 365: part 2

(This is part 2 of a series; see part 1. if you have questions or want to see specific licensing or reporting topics covered here, drop me a line: paul.robichaux@summit7systems.com.)

In part 1 of this series, I talked about some of the license reporting features in Office 365. The big takeaway from that article is simple: many Office 365 admins want to know how many users have installed Office 365 Pro Plus, and on which devices, but the Office 365 reporting tools don't provide a good way to see that data. In conversations with the product team, I learned that yes, they know people want to do this, and yes, perhaps they'll add it in the future.. but it's not on the Office 365 roadmap yet so I am not holding my breath.

As I was working on part 1 of this series, I got a puzzled email from a customer: they had a group of users who had been dirsynced to their Office 365 tenant, but had not yet been enabled for any workloads other than Office 365 Pro Plus. However, some of those users were showing up in the portal with licenses for Exchange Online and Lync Online.

"How'd that happen?" the customer admin asked.

"I don't know," I said... something that I hate saying, especially to customers.

How'd that happen?

This particular customer is unique in several ways. For one, they're using Dell OneIdentity QuickConnect for Cloud Services to perform dirsync between their on-prem AD and Office 365. (Dell seems to have taken a leaf from Microsoft's book of confusing product names, but I digress.) However, all that QuickConnect does is sync directory information; the set of Office 365 licenses assigned to a user is a cloud-only attribute that isn't present in, and thus cannot be synced from, the on-prem AD; ergo, QuickConnect was unlikely to be at fault.

This customer is also using Dell's ActiveRoles Server product to provide delegated AD management. Individual business units use ARS to create and remove users; ARS lets them self-manage their own OUs, then the resulting changes are dirsynced to the cloud. There's an ActiveRoles module that allows delegated administrators to assign Office 365 licenses, too. However, ActiveRoles makes its changes by executing PowerShell cmdlets, so by examining its logs we could see that it wasn't the culprit.

About this time, Mike Crowley mentioned that he had a customer who had the opposite problem: some of their users who had been licensed for Office 365 workloads were showing up in the portal with no licenses. He opened a case with Microsoft support, but the problem vanished on its own for both of us.

I hope you enjoyed this interesting digression. My point (and I do have one) was that something Mike said pointed me at the substance of this article. My second point: sometimes what happens in the cloud is impossible to explain without logs and traces that those of us outside the cloud just don't have.

The many faces of Azure AD

I sometimes struggle with explaining Azure Active Directory to customers, usually in direct proportion to how much they know about AD in general. The thing that trips me up is that Office 365 includes Azure AD, but you don't see it as a separate service. SharePoint MVP Sahil Malik explains it well when he says that "You get an Azure AD when you sign up for Office 365. There is no way around it." See, Office 365 services run on top of AD, just as their on-prem counterparts do. Just as you need AD to run Exchange or Lync yourself, Microsoft needs an AD environment to host these services for your tenant. When you create a new tenant, you're getting an Azure AD partition of your very own… but you can't manage it yourself, at least not the same way you'd manage on-prem AD. You can't see which domain controllers exist, you can't create group policies, and so on. You can dirsync users (and their attributes), and you have some limited admin tools in the O365 management portal, but that's it. Let's consider this the entry-level version of Azure AD: it's bundled with O365 and you're stuck with it. I'll call it the invisible version of AAD.

For no additional cost, you can enable Azure AD Free (clever name, isn't it?) This unlocks some additional features for your tenant, although the feature chart is a little confusing because it says that customizing the logon page and self-service password reset are Azure AD features. For right now, they are, but if you look at the Office 365 roadmap you'll see this:

Sign-in Page Branding enables an Office 365 customer to select custom colors, text and Imagery for their Office 365 sign-in page. Self Service Password Reset allows a user who has forgotten their password to reset it based on prearranged alternative personal information. These two features were previously available with the Azure AD Premium subscription and are now being made available to all Office 365 subscribers.

There are two additional levels of Azure AD, Basic and Premium, that add more features but that cost you money. Basic requires you to have an enterprise agreement (EA), but you can add Premium onto your O365 subscription from within the portal if you need it, which you probably don't.

The key item to notice on the feature chart is the checkmark next to "Basic security reports" in the "Free" column. Let's take a look at how to get started with this feature, what it does, and why you want it.

Enabling Azure AD Free

The enablement process is very straightforward: log in to the Office 365 admin center and expand the Admin triangle, and you'll see Azure AD listed. Click it, and you'll get a new browser window that takes you to the Microsoft Azure sign-up page (shown below). After you fill in the fields, you'll need to click the button in the "Mobile Verification" section to verify your possession of the phone whose number you plugged in. After doing so, you'll see a green "Sign Up" button.

Yeah, like I'm going to put my real phone numbers in a blog post…

IMPORTANT: Before you click the green button, you should know that signing up doesn't cost you anything and doesn't have any impact on your Office 365 service. Apart from seeing some new options in the Office 365 admin center, you won't notice the change. However, as far as I can tell there's no way to revert to the invisible version of AAD, so don't do this unless you're OK with the irreversible nature of this change.

All right, that's out of the way. Click the button already. The Azure factory will hum away for a few minutes as it creates your subscription. You'll get a progress page like this one:

Time to play the waiting game

Note that you may see references to billing for this subscription. There's no billing, because you're using the AAD Free tier. Now you can go back to the O365 admin portal and click the Azure AD link under Admin again. You'll be offered a quick tour of the Azure AD portal interface, which is interesting but not necessary for our purposes; once the tour's dismissed, you'll see the AAD management interface. Click the pyramid icon in the left nav bar to switch to the actual AD management portion of the portal. You'll see a list of each AAD partition that you've subscribed to. Click the right arrow next to the name of the organization, then you'll see a page that looks like this:

While it's ugly, this welcome screen clearly identifies what you need to do next

At the top of the window, underneath your organization name, you'll see a bunch of tabs with familiar-sounding names such as "Users" and "Groups". What you see here is merely a reflection of what was already in your invisible AAD, courtesy of your O365 subscriptions. For example, if you click the Users tab, you'll see all the user accounts that O365 knows about, whether they were dirsynced or created in the cloud.

We're interested in the Reports tab, so click it now.

Using AAD Free reports

As I write this, AAD Free supports six report types, with an additional eight available to AAD Premium subscribers. These reports mostly have descriptive names, which I appreciate; for example, if you click on "Sign ins after multiple failures," it's pretty clear what you're going to see. Depending on what geography your tenant is registered in, you might see a warning dialog that tells you that you're about to see IP address and geolocation information. This is required by privacy laws in some places.

Love the "It is acceptable…" wording.

After you dismiss this dialog… you'll probably get a blank report, depending on which report you chose. Some of the reports have to be generated, and when you first sign up for AAD Free they will not have been generated.

If you're like most of my customers, the first report you'll click on will be the "Account provisioning activity" report. After all, we call the process of creating O365 users and granting them licenses "provisioning," right? It turns out that in AAD-land, "account provisioning" is the process of enabling other web applications to use AAD for single sign-on. When you click on the "Applications" tab in the AAD console, you'll see a list of enabled applications (see below), and you can register additional applications, including Salesforce, Dropbox, Box, Citrix, and even Google Apps (see the gallery for a more complete list). That's what the provisioning report shows: what users have been provisioned with AAD signon for what applications.


Notice that Lync isn't shown because this customer has zero provisioned Lync Online seats

What you probably actually care about—who added, removed, or changed user accounts and licenses—is actually found in the audit report. Below is an example, with the interesting part helpfully highlighted (and some sensitive data elided):

See the red oval? That's the good part.

This report tells me that someone changed the user license for a specific user, and when the change was posted to O365. If that change was unexpected, this would provide a good place to start looking. In this case, the change was posted by the dirsync process, which in this case was expected. However, for changes made manually by an administrator, their user ID would show up here.

Now for the bad news…

While the reports available in AAD Free are quite useful, there are still some problems with their implementation. First off, they are not customizable; there's no filtering, so if you wanted to only see license change events, you can't. (Since you can filter by date, I suppose it's not fair to say there's no filtering). You also can't change the number of days of history: you get to go back a maximum of 30 days.

Second, the reports don't actually show you much data. In the case of audit reports, you get a date/time stamp, information about who made the change, and the bare bones of what changed. It would certainly be nice to have more detail about exactly what changed (even distinguishing between adding a license and removing one would be an improvement).

Third, the reports in AAD Free aren't integrated with the reports UI in O365. This is somewhat to be expected; after all, you can have AAD without an Office 365 subscription (though the reverse is not true). Having said that, it would make things considerably less confusing if reports about licensing were tied into O365.

Finally, and most damning, none of these reports actually answer the question this series started with: they don't tell you which users have actually installed Office 365 Pro Plus! Maybe someday…

About Paul Robichaux