Monthly Archives: November 2011

Team Foundation Build: Sharing parameters between several build definitions

 

Team Foundation Build provides us with a nice editor where we can define parameters (both at design time and queue time) to influence the behavior of our builds and it is really easy to change them.

However sometimes we need to have a parameter shared across several build definitions. Using out of the box features we are forced to place the same value on all build definitions and if we wish to change the value we have to manually change all of them (we can always write some one off code that will replace the value in bulk).

If we want to share a parameter value across more than one build definition we have several choices at our disposal

  1. “hard code” the value in the build definition template, and when we need to change the value we just have to update the build definition template once. Simple, effective but not very visible since the value is hidden inside the build definition (not to mention it cannot be easily changed by non-power users)
  2. Store the parameters in a file stored in source control (for example an XML file or a file with an easy to manage format using a simple text editor). This way when the configuration file is changed every build definition that uses will pick it up immediately. This requires that you change your build definition to get the file from source control (you can use the built in activity DownloadFiles) and read the parameters from the file. This also has the traceability advantage  (by having the parameters stored in source control, we have history, auditability and the ability to easily revert to previous versions).
  3. Store the configuration an a SQL database. For this you can write your own activity or use the SQL Server activities from the Community TFS Build Extensions
  4. Store the configuration on an external system accessible via Web Services (WF has built in support for this)
  5. Store the configuration in the TFS Registry, I’m not talking about the windows registry. I’m talking about the TFS registry, which is a feature that allows you to store settings (in a hierarchical manner) on a registry. There are two scopes for the registry, a collection registry or a server wide registry (also known as the configuration registry). The advantage of the registry is that it’s stored within TFS. Everything is integrated, when the database is backed so are your settings, everything is self-contained. The disadvantage of this approach is the fact there are no tool (out of the box) to read and write settings into the registry.

This post is all about option 5), to read values stored in the registry from your own build definition templates, you can use an activity that I’ve just committed to the Community TFS Build Extensions. The activity is called GetValueFromRegistry

SNAGHTML42e8aec4

To use the activity all you have to do is drag it to the canvas and provide two parameters (one of them being optional).

image

The Path parameter is mandatory and represents the path of the value you wish to read (for example TFS email notification address (if notifications are enabled) is stored in /Service/Integration/Settings/EmailNotificationFromAddress)

There are no rules regarding the use of paths, but it is wise to use a convention (but using /Configuration or /Service is not recommended).

You can use for example /Extensions/<CompanyName>/  as the base path for all your key values.

The  Default Value parameter contains the default value that will be returned if the value is not defined (or if the user doesn’t have permission to read it).

The registry value or the default value will be returned on the Result parameter.

All parameters (including return) are strings.

Please notice registry reading as a little quirk. If the user doesn’t has permissions to read a given value no error will be returned, but no value will be returned either. So if the build user doesn’t has permissions to read a given value it will return the default value (or an error if no default value has been defined, see error below). Which means the build may fail if you can’t read a value and if have set the FailBuildOnError parameter to true.

image

The activity will first attempt to read the value from the collection registry, if no value is found then the server registry is attempted and if not found it will returned the default value (if defined).

Note: this activity is not yet available on the latest released version (v  1.1 released on September 2011), so if you want to use it you will need to download the source code and compile it yourself or wait for the next public release (v 1.2) that is scheduled for December

If you want to learn more about the TFS Registry, some of its semantics and how you can store values on the registry you can read more about on these two posts:

Using Team Foundation Server Registry–Part I The concepts

Using Team Foundation Server Registry–Part II the API

Thanks to Mike Fourie  for proofreading and suggesting some improvements to this post.

Using Team Foundation Server Registry–Part II the API

 

On the first part of this series I talked about what is the registry and some basic scenarios usage, on this post I want to talk about how you manipulate registry entries.

You have two choices, using an utility to manipulate entries or using TFS object model (or SDK as it is officially called) using a .Net language or Powershell

Using an utility

Neno Loje wrote tfsreg.exe that you can use to manipulate (both read and write) registry values.

You can see an example of how to use it on a blog post  where Neno explains how to change the email notification delay (a setting that is stored in the configuration registry)

You can download the tool from the mentioned post.

Note: Although it is not clear on the post or on the utility itself, this tool only allows you to manipulate registry entries from the configuration registry (at least for now Smile) .

Reading/Writing values

All code is using the client object model, but most (if not all) calls also have a counterpart server object model that can be used on server scenarios.

You can manipulate registry entries using the ITeamFoundationRegistry interface.

Assuming you are already connected to TFS and have in your possession a valid reference to

TfsTeamProjectCollection object all you need to do to get a value is this (assuming you have enough permissions)

ITeamFoundationRegistry service = tfs.GetService<ITeamFoundationRegistry>();
 
service.GetValue(key);

You can also have at your disposal the generic methods GetValue<T> there are also overloads that can return a default value if the key is not found (OR you don’t have permissions to read it)

To write a value all you need is to use the SetValue member.

Pretty simple huh?

This is to access the a project collection registry, if you want to access the configuration registration instead you (if you have a connection to a project collection) you can get a reference like this

var regService = tfs.ConfigurationServer.GetService<ITeamFoundationRegistry>();

All the rest remains equal (you can also connect directly to the server using the class TfsConfigurationServer and then obtaining a reference to ITeamFoundationRegistry

Setting Permissions

Setting permissions requires a bit of more work,since there are (that I know of) not utilities to do this. It is still simple anyways. In order to read/set/remove a permission you need to use SecurityNamespace to manipulate objects from Registry Namepace, we can obtain it  via ISecurityService.

So in order to get a security namespace that can manipulate permissions from registry objects you need to something like this snippet:

var server = tfs.GetService<ISecurityService>();
var securityNamespace = server.GetSecurityNamespace(FrameworkSecurity.RegistryNamespaceId);

We can then use the SetPermissions method to set out permission, the following snippet

securityNamespace.SetPermissions("/", identity.Descriptor ,RegistryServicePermissions.AllPermissions, 0, true);

Now to dissect these parameters.

The signature for the method is:

public abstract AccessControlEntry SetPermissions(
    string token,
    IdentityDescriptor descriptor,
    int allow,
    int deny,
    bool merge
)

 

  • token paremeter we will pass / which means we will be granting access to the entire registry. If you want to grant just access to a part of the tree, this is the place to do it
  • identityDescriptor we will pass the account/group for which we will be granting the parameter (more on this later)
  • allow parameter we pass the value the value RegistryServicePermissions.AllPermissions since we want to grant read/write access
  • deny parameter we will pass 0 since we don’t wan to deny anything
  • merge we will pass true, since we want this permissions to be merged with existing ones (just in case)

Note: the value 1 and 2 on the allow/deny parameter depends on the type of security namespace we are dealing with, and it can be obtained from the security catalog.

To get and identity descriptor you need to use IIdentityManagementService, this snippet will allow you to get the identity of the group [DefaultCollection]\Project Collection Build Service Accounts

var identityService = tfs.GetService<IIdentityManagementService>();

var identity = identityService.ReadIdentity( IdentitySearchFactor.DisplayName, @"[DefaultCollection]\Project Collection Build Service Accounts", MembershipQuery.Direct, ReadIdentityOptions.None);

Notice the first parameter is key to the search, you can either search the user by it’s display name or by account name (IdentitySearchFactor.AccountName), but if you use Account name, the account will have to be in the format

vstfs:///Framework/IdentityDomain/<ID>\\<Name>

so using the same example the account name would be

String.Format("vstfs:///Framework/IdentityDomain/{0}\\Project Collection Build Service Accounts", tfs.InstanceId.ToString());

It is very important that specify the searchFactorValue in the right format or you won’t find the identity descriptor you are looking for.

Using Team Foundation Server Registry–Part I The concepts

 

Since the beginning Team Foundation Server uses a registry to store it’s settings (like Sharepoint location,external address,database configuration, etc), with the introduction of the 2010 version, the registry has been revamped to be much easier to use (pre 2010 it was a XML file that you had to manager yourself. Including the values that didn’t pertain you).

So starting with 2010, we have a nice API to manipulate the registry in a easier way. The registry is not only used by TFS to store it’s settings but it can also be used to store your settings.

There are many scenarios for which you can use the registry. If you need to store metadata that is project related and needs to be shared globally among several users you can use the registry to store those values instead of storing them on your own database.

The advantages are obvious, no need to manage an external storage mechanism, the data is contained within TFS with all the associated advantages (backup/restore, if you move a collection to another server they follow the collection, etc.).

The registry is not a database, you don’t (formally) define a schema for your settings. The registry is an hierarchical key/value database. You can define your keys in an hierarchical manner and the registry is agnostic about your data values (so you can use it to store, .Net types (being string the most common) or XML).

For example TFS uses the registry to store the email notifications settings on the following keys:

  • /Service/Integration/Settings/EmailEnabled
  • /Service/Integration/Settings/SmtpServer
  • /Service/Integration/Settings/EmailNotificationFromAddress

Notice the trend? Service –> Integration –> Settings stores a bunch of TFS related settings.

The API is public and you are free to use the registry, unfortunately the registry is not widely documented. I hope with this post demystify the registry, it’s semantics and how you can manipulate it.

There are two types of values in the registry. values and user values.

The values are registry scoped and anyone can read/write them (as long as they have the appropriate permissions), so it is appropriate to use when you have a setting that is going to be shared among all users.

User values are scoped to the user, they are private to the user and only the user can read/write them. This is great if you want to store settings for a user independent of it’s location, so you can store values in the user registry and he/she can roam around machines having the same settings everywhere (when you say application you can also say Fat client (eg: visual studio) and web. In fact this is exactly what Web Access does, it stores user preferences in the user registry.

There are two different registry storages, the project collection registry and the server registry (stored on the configuration database). The collection registry is stored along with all data of the project collection, so if you detach the collection and move it to another server the registry follows the collection, the configuration registry is stored on the configuration database, so you should use it to store parameters that are server related. If you move collection to another server and you are relying on some values on the configuration registry, it will be your responsibility to recreate them on the other server.

There are two kinds of security constraints applied to the registry. To read/write registry (depending on the registry) enters you need the following permissions

  • View collection-level information for a collection registry
  • View instance-level information for a configuration registry

These permissions are the first barrier, then you need permission to access the object itself (but since this is hierarchical you can grant/deny permission to a branch of tree). By default these are permissions to registry keys.

For the configuration registry

Permission Path Read Write
Team Foundation Service Accounts

/

X

X

SharePoint Web Application Services

/

X

 
Team Foundation Administrators

/

X

X

For a collection registry

Permission Path Read Write
Project Collection Service Accounts

/

X

X

Project Collection Administrators

/

X

X

Project Collection Valid Users

/Service/Registration/RegistrationExtendedAttribute/

X

 

Note: reading the registry has a semantic that is not common, if don’t you have first level of permissions (View collection-level information for example) the operation will fail with an exception. If you don’t have permission at the object level any read operation will not fail with an exception but it won’t return anything (even if the object exists).

This happens to avoid disclosing information that a node exists (it’s mere existence) to people who do not have access to it. It’s a typical technique of multilevel security systems to avoid leakage of information with inference atacks. For example if you have a value with a key /Settinga/OnToBeFiredList/JohnDoe I may not have access to it’s value due to lack of permission but if an error was flagged we could know it exists. Sometimes the mere acknowledge of the existence of information is reason enough to be denied.

It is debatable if the existence of such semantics doesn’t out weight the costs of not having errors being flagged when we don’t have access to information that we want to access. But such discussion is pointless since that the way things are. Smile

As of today (to be read, it is like it is today it can change tomorrow) access to the registry on Team Foundation Services (AKA as tfspreview.com or TFS on Azure)  has been hardened so you can’t read/write no matter what kind of permissions you have.

On the second part of this series I will delve into the registry API, how we can read/write values to the registry and now we can manage permissions to it.