“Thinking about design is hard, but not thinking about it can be disastrous.” – Ralph Caplan

Introduction

For almost 14 years in the ServiceNow space, and across a rapid expansion of the exosystem, it has been interesting to observe and analyze various organization’s approaches to developing and maintaining their ServiceNow environment. Specifically, how do organizations manage the inflow of business needs, the distribution and velocity of development, configuration and administrative work, and the ongoing maintenance of the platform? As the footprint of ServiceNow has expanded conjunctionally with the expansion of the business functions the platform supports, I increasingly see divergence in these strategies. I’m writing this article to articulate these strategies and provide my view of how each succeeds and fails, along with my recommendations for the correct strategy given a company’s view of ServiceNow.

Management Approaches

There really are two ends of the spectrum for how ServiceNow environments are managed. At one end is the idealized view and what ServiceNow itself espouses: Use of Idea and Demand Management to receive and vet business requirements, an oversight board – which ServiceNow calls a “Center of Excellence” – who does the vetting and prioritization of these requirements, Agile and PPM to manage the development work to fulfill these requirements, and an operational organization that handles release management, break/fix work, upgrades, and performance and security of the platform. If you’ve got your thinking cap on while reading this, you’ll quickly sense that this is intended and works best in the largest ServiceNow implementations, where the platform is a large part of an overall enterprise strategy for a company.

The other end of the spectrum stems largely from traditional IT functions; that is, a purely operational model and mindset where all development is treated as one-off break/fix/enhance type work. Where “keep the lights on” is the primary and sometimes only strategy. In these organizations, ServiceNow development work is typically handled through Incident and/or Enhancement processes, and each task is designed, developed and released “in a silo”, usually without thought to larger strategic initiatives. In other words, the view of the development does not extend beyond the scope of the need elucidated.

With a 25 year career in IT, I’m certainly aware of and sympathetic to this mindset. I find it particularly prevalent in MSP or MSP-like organizations. It’s not that the people running these organizations intend to be “unstrategic” (not a word), it’s what they know. These mindsets are built over years and decades of running IT as an operational entity.

There is a cost to doing business this way – and this is the crux of this article. When you implement under an operational mindset, you necessarily build everything as one-off. Critically, there are no design or architecture considerations taken into account, which means there are concerns for platform maintenance, stability, health and optimization. These can range from the simplest quirks like inconsistent forms and fields, and re-creations of code logic, to large-scale issues with performance and user experience.

Examples

Here are some specific examples of development done without design or architecture prior to “hands-on” work:

• A developer customizes an out of box ServiceNow application when a custom application would have served the requirement better. This leads to upgrade issues.
• A developer builds a security requirement using client-side functionality, which is pseudo-security. This security hole is exposed when using integrations and server-side functionality to the same data.
• A series of requirements for a single application are developed as one-offs. After these are implemented, the UI/UX experience is compromised, as now the form design is cluttered and out of sync with other applications. Adding UI logic and many related lists hinders the performance of form loads.
• One developer uses Workflow, another Flow Designer, another a series of Business Rules, and another a series of Glide Ajax client scripts, all to implement similar requirements. Maintenance becomes hyper complex as each change or fix is a one-off “head scratcher”; “Now where is this functionality??”

I can argue that Agile is a contributor to this problem. Not the methodology itself, but the incomplete usage of the methodology. I often see organizations going no further than Stories to manage their work. While Stories done correctly are ideal for “state the business requirement, not the technical how” of documentation, without using Epics to group requirements into a cohesive release, and more importantly, without architectural design overseeing the Epic, the Stories are developed in silos and lead to the issues noted above.

Best Practice

In my experience, the best practice is to have an architectural or design review of all requirements prior to development. Some may only need a cursory review to confirm it can be built as a one-off. Others may need a directed technical approach because there are several ways it could be built, and a consistent approach platform-wide is best for long term maintainability. And some may need a complete analysis of pros and cons, buy versus build, custom application versus out-of-box in order to build the right solution for the business need and the platform sustainability.

I’ve included a diagram below that shows the “idealized” process flow, including a Center of Excellence that fulfills this function:

The concept of a Center of Excellence, or at least some level of architectural oversight, is not meant to be onerous or a process bottleneck. This is a common concern organizations have, and a reason they don’t do it. My argument is that the operational sweet spot for such a function lies somewhere in the middle of the spectrum: Organizations are free to be as fast and independent with business requirements as they choose. The oversight part of the process is a simple architectural design review of all Stories (requirements) prior to development, with the end result a proper technical approach to development. A good architect will quickly recognize when there are multiple approaches to implement a requirement and provide guidance on the correct approach, taking into consideration all aspects mentioned previously. If the Agile methodology is being used, this can be part of the grooming process.

The diagram above is one I drew for where the Center of Excellence lives in the overall development process, between the requirements gathering, and the execution, either as operational one-offs or as larger project-type initiatives.

ServiceNow’s Documentation on Center of Excellence

In the end, it comes down to an organizational decision, even if not made consciously: “Do we spend the time up front to ensure things are developed in a cohesive platform strategy way, or do we dedicate more time, money and resources to fixing issues when they (inevitably) rise in Production?”  The simple analogy is working to prevent forest fires, or dedicating resources to fighting forest fires.

It’s been a few years since I wrote Part 1 of this article, going through the history and evolution of the ServiceNow platform, and the morphing of the company strategy from platform to product. After working with multiple clients in the meantime, and reading lots of new marketing and going through many platform release upgrades, I thought it time to revisit the subject with new perspective and analysis.

A quick recap: In the early 2000s, ServiceNow (nee “Glide”) was envisioned and built as an extensible business workflow platform, designed to replace paper and manual processes, but without a defined business application built-in. The idea was that businesses would analyze their own processes and automate using the platform components. Once this didn’t catch on, the SN founders built an application suite on top of the platform using what they knew – ITSM. This caught on, and in the subsequent years both customers and ServiceNow have used the extensibility of the platform to build and solve countless business processes problems. As ServiceNow itself has gone public and had multiple leadership changes, the company has shifted development, sales and marketing focus to products it builds on top of its own platform. This is why most discussions around new releases are around Products, and not platform enhancements.

While this all may be natural progression for a company that goes public and has to answer to the markets and short-term financial interests, it poses some issues for those attempting to use ServiceNow as a platform rather than a series of products.

Buy versus Build

In the early days of ServiceNow, the process for implementing a business process solution was generally straightforward (other than specific ITSM processes, which were built in). As a consultant, you would listen to the business problem that needed solving, then design and implement a custom* solution using the platform components provided (see Part 1 for more detail). There wasn’t a longer discussion or decision required for build versus buy, since the platform was designed to be built upon. ServiceNow provided the components to build the tools (applications) to solve business problems, and the licensing was based on fulfiller versus requester. There was no further buy versus build decision to be made.

*NOTE: Although this could spawn an entirely separate discussion, I want to point out that in the ServiceNow world, “custom” is not a bad word, though it is often seen as such.  In reality, building a new application using the platform components ServiceNow provides is doing exactly as the founders intended.  It is also not really “custom” in the true sense of the word. It is simply a new way of using the components provided.

Simplification by Obfuscation?

The nature of strategizing control over flexibility means you take the power of the platform out of the hands of those who are best equipped to take advantage of it. This has been true long before ServiceNow and will continue to be true long after, but I believe it is exacerbated in the ServiceNow space by the factors previously mentioned: market forces, management changes, market strategies, misinformation and misconstrued information. Over the years as ServiceNow has moved from a ticketing system to a strategic platform for companies, I’ve watched as C-level executives have injected common phrases like “stay out of the box” and “minimize upgrade efforts” into the lexicon. I can only assume these come from history with other platforms and reading industry studies, rather than from deep knowledge of what this really means for their ServiceNow implementation. I also assume because those who are making the financial decisions are saying these things, that they become both gospel and strategy for those who have a vested interest in their decision-making.

I liken what ServiceNow has done to the platform to using WordPress for website development, rather than DreamWeaver. The latter is a framework that gives you pre-built components that experienced developers can use to build custom websites faster. The former is more for non-developers to implement fully pre-built websites with a small to moderate ability to make configuration changes. But for an experienced website developer in certain circumstances working with WordPress can be more challenging because things that could easily be modified in CSS aren’t always accessible. In this way, WordPress makes it easy to deploy a website that fits their model, but makes it far more difficult to make what are often necessary changes after the fact.

Business Example:

Here’s an example of what I’ve been describing:

Business Requirement: A need to manage company events such as luncheons, meetings and guest visits. The company wants to use their ServiceNow investment and the platform tasking and workflows to do so.

Platform Solution: A ServiceNow architect uses the Task application model to extend to a new Task Type called “Event”, creates a Record Producer to intake customer needs (available in the Service Catalog), builds sub-tasking records and initiates them with a workflow based on state changes to the parent Event. Form and list views, notifications and reports are configured to meet business needs. Security for the new application is layered on as needed. Any specific business requirement can be implemented without concern for “breaking” out of box solutions, and is completely upgrade-proof (ServiceNow doesn’t care about net new applications and components – they are completely ignored in upgrades).

“Out of the Box” Solution: The customer ServiceNow team is told to stay “out of the box” and so attempts to build the solution in Service Request with a Catalog Item for intake. The Event data takes the form of many variables on the Requested Item. The workflow is driven off of variables, and Catalog Tasks are initiated by the workflow. The ServiceNow team has to customize the Request and Task forms for Event needs, creating maintenance issues – the application looks and functions one way for “normal” Service Requests, and a different way for Event Requests. Forms, lists, reports, notifications, security are all doubled with mutually exclusive conditions. Subsequent implementations like this use case in Service Request further complicate the configurations and maintenance.

Product Solution: A ServiceNow Account Manager hears “buzzwords” from the customer regarding their business needs and finds an out of the box product to license them. The customer installs the new product and demos for the business. Stakeholders find that the product only partially aligns with their needs. The business has to make a decision to either customize the ServiceNow product for their needs, or go through a rigorous and costly OCM cycle to change the way their business works to fit the ServiceNow product. If choosing the former, the company loses the ongoing maintenance benefits of staying “out of the box”, while still paying new licensing charges. Anyone who has worked in corporations knows the latter requires an incredible sales job to accomplish – businesses DO NOT like to change!

If you’re intuiting the conclusion I’m reaching with this example, you realize the irony is that what most would call the “custom” solution is actually the solution with the least development friction, the least technical debt, and the least upgrade concerns.

Conclusion

We’ve reached a concerning level of misinformation and mischaracterization of design and development decisions as ServiceNow has changed both their platform focus and marketing strategy. But what ServiceNow cannot change is the fundamental nature of the platform any product they build and market is based upon. Those architects and developers who truly understand this fundamental nature are much better equipped to deliver real value to their customers via shorter development cycles, maintainability, and upgradeability. Remember: “custom” is not a bad word!

In journalism, there’s the concept of the Five W questions whose answers are fundamental to getting the information needed:

• Who
• What
• When
• Where
• Why

I want to talk about what I call the “Three Ws of Development” in the ServiceNow realm. These three are: When, Where and Why. We’re going to skip the questions “Who” and “What”. Why? Because “who” is a question for hiring managers, recruiting, and resource vetting. And “what” is (too often) the focus of most if not all training and documentation. Do you need to get the current user’s ID on the client side? Check the API – that’s the “what”. Instead, I want to focus on some areas of development consideration that I feel are often neglected, and I’ll explain each and try to put them in context.

Most everyone in the ServiceNow world knows the basic system architecture, which is the same for almost all cloud-based applications:

On the ServiceNow side, there’s an Application Server that stores the compiled code, the files (images, etc.) needed by the application, and delivers content to the user’s browser when requested. The App Server connects to a SQL Database that stores all the data records, and in ServiceNow’s case, all the configurations and code customizations created by both ServiceNow and customers.

Now consider a “normal” transaction between these entities. We’ll use one that’s fundamental to ServiceNow: Accessing and updating an Incident record. The following shows all the component parts in this transaction:

1. Client makes a call to the server to get a specific Incident record
2. Query Business Rules determine if certain records should be restricted from view
3. Application queries the database for that record from the incident and related tables
4. Database returns required data to the application server
5. ACLs are evaluated and applied
6. Display Business Rules run logic and place data on the scratchpad (to use client-side)
7. Application Server provides browser with form, data, and all configurations based on 4-6

8. onLoad UI Policies and Client scripts run
9. User manipulates data, onChange UI Policies and Client scripts run
10. UI Actions run client-side, server-side, and sometimes both (first client, then server)
11. On Save, client sends form data to the Application Server
12. Before Business Rules manipulate record data prior to saving to the Database
13. Application Server sends data to the Database
14. After Business Rules run on any table (current, previous still available)

This is a broader “order of execution” list than ServiceNow provides in their documentation, which deals strictly with Business rules.

So how does this apply to our Ws discussion? Let’s discuss:

Where

In considering where to develop your configurations and customizations, you will almost always get better performance having them run on the Application Server rather than the client’s browser session. Observe the middle section of the diagrams above and the components that live in it. For example, when a user accesses a record, if you need to run logic and have the result available on in the client’s session, it is faster performance-wise to run the logic in a display Business Rule and place the result in the scratchpad rather than run the logic in a Client script, particularly if the latter would require querying the database after the record has been accessed and loaded in the client browser session.

Another important use case is security. All security should be done in ACLs, and only supplemented with UI Policies and scripts where needed. ACLs are the only true security, all other methods are posing as security and can usually be bypassed. For example, let’s say you have a Social Security Number field on a User record that should only be available (visible) to Human Resources, and should not be editable by anyone (it feeds from an HR/HCM system). This field should be secured with field-level read ACL for users with an HR role, another field-level read ACL marked false for all other users, and field-level write ACL marked false for all users. If you were to use a data dictionary level read-only marker, this could be bypassed by scripts running as the system. If you were to use a UI Policy or a Client Script to make it visible and/or read-only, this could be bypassed by list views and edits, server-side scripts, and Integration APIs.

In keeping with the last idea, it is also good practice to mimic your client-side logic on the server-side. For example, if you don’t want to force a field to be mandatory for every save, but want to run client side logic that prevents the save of the record without the mandatory field in a particular scenario, you should also create a server-side Business Rule that aborts the save and messages the user about the mandatory field. This way, your logic is enforced in list edits and server-side record manipulation, and not just in form views.

• List edits bypass “standard” Client Scripts and UI Policies
• External systems integrate directly with the Application Server

Note: You may have noticed that I haven’t mentioned List configurations like List Edit client scripts. These are also usable to “fix” the List edit issues mentioned, but it doesn’t fix server-side logic and integrations.

When

Going hand-in-hand with the Where is the When of development. Specifically, consideration should be paid to when in the lifecycle of the full transaction the development is best to exist. Consider the following Business Rule scenarios:

• You need to access information client-side that is not available on the current record being accessed but can be ascertained using the current record (e.g. querying information from another table using field data from the current record. The correct time (when) to run this logic is in a display Business Rule and place the needed information in the scratchpad.
• You need to manipulate field data for the current record before the record is saved to the database; for example, you need to multiply two integer fields from the current record and store the product in a total field on the same record. The correct time (when) to execute this is in a before Business Rule.
• You need to manipulate field data for a different record using the current record data, and either need to access the “previous” object, or you need the manipulation to be reflected in the user’s updated client session view. For example, you are updating a related record to the current record and the update needs to be reflected in a related list of the record form the user is currently viewing. The correct time (when) to execute this is in an after Business Rule.
• You need to manipulate field data for a different record using the current record data but it doesn’t need to be reflected in the user’s updated client session view, OR you need to trigger an integration whose updates don’t need to be reflected in the user’s updated client session view. The correct time (when) to execute this is in an async Business Rule. Async Business Rules create a Schedule Job that gets queued and executed as system resources become available.

Why

There are many other scenarios an experienced system admin/developer can think of. A key is to understand all the component parts of the transaction diagrams above; the goal is to configure and develop in the correct place, both where and when. Why? The benefits of developing in the correct time and place are:

• Performance: Following the when and where guidance will ensure the best system performance. Running logic on the server is almost always faster than on the client. Running logic asynchronously means the client, and therefore the user, means they don’t have to wait for the transaction to complete. Each time the transaction can avoid another database query means faster execution and less wait time.
• Security: Developing measures that secure fields and records, both visibility and editability (R and U in CRUD), in the correct place, means your instance is actually secure, versus appearing to be secure to an end user. After all, most if not all security breaches do not come from a human clicking and typing.
• Maintainability: Broader maintainability comes from following good and best practices. Consider a ServiceNow health check – what is the likely result of a system audit if you follow the practices suggested above versus using whatever means are available to create solutions.

Conclusion

ServiceNow provides many means to an end. Sadly, much of the documentation and training does not cover which the correct means to the end; rather, it simply details, “If you need to do X, go here and do this”. It doesn’t say why you should do it this way, or what the overall implications are to doing it that way. What I’ve tried to do is give you a baseline of how the system works, so you can understand where and when the correct places are to do X. Understand the baseline, and stop and think about it before everything you develop. I promise your platform, and hopefully your work life, will be better for doing so.

Happy developing!

Over the past few years, there’s been a movement in the business world to espouse the concept of “citizen development”. Gartner defines a citizen developer in this way:

A citizen developer is an employee who creates application capabilities for consumption by themselves or others, using tools that are not actively forbidden by IT or business units. A citizen developer is a persona, not a title or targeted role. They report to a business unit or function other than IT.

Gartner: Definition of Citizen Developer

ServiceNow has espoused this concept in its marketing materials. You’ll see any number of websites, references, articles and online sessions talking about the need for citizen development and how ServiceNow supports it. From their website:

ServiceNow offers a series of citizen development tools, from low code to no code.

• APP Engine Studio and Templates
• Flow Designer
• Process Automation Designer
• Integration Hub
• Virtual Agent
• Predictive Intelligence
• Performance Analytics

ServiceNow: What is Citizen Developer

For the purposes of this article, I’m going to assume that citizen development is marked by these important aspects:

• The development does not involve writing code, particularly logic constructs
• The development does involve GUI elements that allows drag and drop construction
• The developer does not need to be concerned with version control or other development processes
• The development does not involve environment promotions or other release management aspects

Based on the blurb above, I’m going to assume that citizen development in ServiceNow (the platform), as considered by ServiceNow (the company), can involve any or all of the following:

• Flows using Flow Designer
• Table creation
• Data elements (fields)
• Data dictionary attributes
• Form builds
• UI Policies
• Catalog Items

I’m sure I’m leaving quite a bit out, but I wanted to hit the “big stuff”. To me, this list encompasses the things Fred Luddy envisioned when the platform was first devised and built in 2004 (Tables, Fields, Forms, Policies) and the current marketing push around Flows to control all business logic.

Ultimately, what I think ServiceNow is envisioning is the ability to take a whiteboard or Visio-type flow and turn it into actual configuration in the platform, particularly by folks who aren’t developers by trade. This is certainly the Flow aspect. Additionally, the “Fred stuff” included being able to develop new data entry forms and business logic around them to digitize what was previously done on paper, email or spreadsheets. (Really, when you boil down ServiceNow to its core value, this is it.) Also done by “non-developers”.

Here’s the fundamental flaw in this thinking: This level of development democracy assumes the guardrails are big enough and strong enough to prevent the introduction of flaws into the system. These are simple flaws – inconsistency in form design or label nomenclature, and large flaws – applications that don’t fit platform architecture standards set by either ServiceNow or the platform owner(s). So what are the guardrails? ServiceNow will tell you Application Scoping, which silos application elements by namespace and security. However, if you are a platform owner – responsible for the overall health of the entire system – what good are silos if every silo contains a giant mess? It reminds me of the closet scene from The Marx Brothers “A Night at the Opera”.

I certainly have not seen cost-of-maintenance studies done on this, but I have a hard time imagining that the cost of “cleaning up” and/or maintaining the applications developed by non-developers is less than development that uses an architecture, a methodology, and developers.

Consider this analogy: If you were going to try and develop a cogent political philosophy, would it be wiser and easier to read the missives of a few political experts across a range of theories, or turn to social media and attempt to cull the same from it? I often think of social media as citizen development. It’s the “everyone contributes and we hope something cohesive comes out of it” mentality. If this has worked, I’ve yet to see it happen.

I think of another analogy from my college days: In my Operational Management class, specifically the time focused on lean manufacturing, my professor told a story of visiting various auto manufacturing plants. When he visited the Mercedes-Benz factory, he told of a team of highly paid, highly skilled engineers who sat at the end of the assembly line and reviewed and fixed every defect in the car before it passed inspection. So when a consumer buys a Mercedes-Benz, they know for an expensive price, they are getting a quality car. (We’ll leave the cost of engineering and parts out of the equation for now.) Conversely, when he visited a Toyota factory, he saw that every line worker had an alarm they could ring and stop the assembly line whenever they noticed a potential defect caused by anything in the system – process, parts, etc. The lean manufacturing methodology accepts upfront costs of line stoppages in order to eliminate the possibility of downstream defects and the cost of correction. The moral of the story is each way results in a quality product, but the way to get there is completely different.

To come full circle, I see citizen development as choosing the Mercedes-Benz way – where the acceptance of inherent defects means choosing your developers’ efforts will be on the back-end. I say this because, in my experienced opinion, I’ve not seen where the result of citizen development is clean, designed to platform standards, defect-free, and simply able to be deployed with no concern. Conceptually, there likely are ways that putting up enough guardrails is possible, but I don’t think it’s possible in the ServiceNow platform today. Specifically, here are the things required for development, or related to citizen development in ServiceNow, that are outside of the abilities of non-ServiceNow developer:

• Understanding how an application should be built given a platform architecture strategy.
• Where a new table should live in the database structure.
• Deciding what form views should exist and whether they should be forced.
• Any management of record security.
• Any coding. If you’ve worked with ServiceNow, you know there are plenty of business logic aspects that cannot be implemented without some code.
• Troubleshooting Flow Designer bugs. As of Quebec, Flow Designer is still buggy and simply doesn’t work as expected.
• Working with variables on Catalog Items. Since Catalog Items are a big aspect of the ServiceNow citizen developer push, it implies that a citizen developer knows how to define the correct variable types, when to use variable sets, how to do advanced reference qualifiers, and all the other “developer-y” stuff related to variables.

As I said many years ago at a Knowledge conference when the concept of citizen developer was introduced, “Ask a citizen developer when they should or should not extend the Task table.” I’d love to be proven wrong on this, but I’m still waiting.

Happy developing!

Application scoping protects applications by identifying and restricting access to application files and data. By default, all custom applications have a private scope that uniquely identifies them and their associated artifacts with a namespace identifier. The application scope prevents naming conflicts and allows the contextual development environment to determine what changes, if any, are permitted.

Source: Application scope

Each ServiceNow application has an application scope that determines which of its resources are available to other parts of the system. Application scoping ensures that one application does not impact another application. You can specify what parts of the application other applications can access by setting application access settings.

Source: Understanding Application Scope on the Now Platform

The concept is that application code, configuration and data is programmatically “siloed”. This is the ServiceNow version of the old encapsulation concepts in object-oriented programming (OOP). In ServiceNow, applications across scopes have methods of interfacing with each other, but the elements that make up the application (tables, UI element, scripts, UI configuration) cannot be directly accessed by other applications unless granted specific permission to do so.

An important note: When ServiceNow moved to application scoping, all existing platform applications and elements moved into the “global” scope. This is still a scope, but with special properties – it is by its nature more accessible. In general, by default any other application can access global elements and data, rather than by default not being able to access.

Since its introduction, ServiceNow has been touting application scoping as a panacea to many development ills. The stated “best practice” is to always build new applications inside of a scope, and if you’re familiar with installing new applications from plugins or the ServiceNow store, you’ve likely seen quite a few scopes added to your platform environments. For example, in my Personal Developer Instance (PDI), which has very few plugins installed and no Store apps, I have 115 primary application scopes. (The main thing I’ve installed outside of the baseline platform is Customer Service Management.) To me, ServiceNow has gotten “scope crazy”. For example, in my PDI, just within the Change Application there are these scopes:

• Change Management – Approval Policy
• Change Management – ATF
• Change Management – CAB Workbench
• Change Management – Change Schedule
• Change Management – Color Picker
• Change Management – REST API

I can see having Change Management within its own scope. But why are six (6) needed? Does a change approval policy need to be in its own scope? If you’ve tried to make some basic configuration changes within an application, like a reference qualifier or a dictionary override, you know what a pain it can be to have to constantly change scopes and manage all the resulting update sets.

So why has this change been made? Here are some of the stated benefits of Application scoping:

• “Containerization”: All aspects of the application are contained in a namespace
• Security: By default, parts of the platform outside of the application have limited to no access to the application
• Versioning and governance: The entire application has a version, versus the individual elements
• Deployment: Much like versioning, the entire application can be deployed and removed as an entire entity
• Isolating development: Because of the above, large development organizations can segregate development by application

Sounds wonderful, doesn’t it? I’m in total agreement with the concept. However, there are several known and stated qualifiers to developing this manner (in ServiceNow):

• If you are building an application that extends from an existing application in another scope – including global – any element you want to leverage has to be copied into your application scope. For example, if you need a Business Rule’s logic from global, you have to copy the whole Business Rule to your application scope.
• There are different platform APIs for global versus scoped applications. Many of the API calls are the same, but not all of them are. You have to be sure you are looking at the correct API documentation for your current use case.
• Managing development can get more complicated – you’ve got to pick the correct application in addition to the correct update set. This gets more complicated if you are developing across scopes.

Additionally, I’ve noted some other concern in my experience working with this functionality:

• By isolating development, there is a human nature aspect to consider – that developers are less likely to consider platform architecture when developing. There may be a comfort level with the siloing which might lead development organizations to skip oversight of design and standards.
• Having to develop so that applications can use cross-scope APIs and elements adds a non-trivial amount of overhead to development times. In my experience, I’ve noticed a 20-25% uptick in time to develop inside of non-global applications. This is not an across-the-board universal uptick; rather, most things take the same amount of time, and once in a while something that would take 5 minutes in the global scope takes 2 hours in the application scope, as you scratch your head and desperately search why something that works quickly and easily in global doesn’t work at all in your application scope. So there is a large uptick in time on a small number of elements that averages to a 25% total time increase over the course of a development project.
• Pushing upgrade sets across environments using multiple application scopes causes more conflicts and errors than just using global.

My conjecture is that had ServiceNow been designed with scoping from day 1, these probably would not even be concerns – it would have been worked out and eliminated in the platform design process. Instead, scoping has been bolted on “after the fact”, and it raised these and other concerns I’m probably forgetting.

I’ve thought a lot about the question of whether to scope or not to scope. I get asked about it all the time on projects that require custom applications. In my experience, these are the actual reasons to use an application scope:

1. You are building something you want to package to put on the ServiceNow Store or Share site, outside of your ServiceNow environment(s).
2. You save money on licensing.
3. You aren’t versed enough in the platform to feel comfortable implementing isolation techniques in the global scope, such as ACLs, or you aren’t comfortable with your developers for the same reason.
4. You work in a massive development house with many teams developing in ServiceNow without any centralized management of the design, architecture and QA/QC of the platform.
5. You want to be able to say you followed ServiceNow’s advice.

What if these don’t align with your needs? My advice is that you do not need to create new application scopes in order to do custom development. Do what you’re comfortable with. I prefer the ease of developing in the global scope, and I much prefer architectural oversight of the development happening in the platform. When I consider a set of business needs, and determine custom development is required to meet them, I want to be able to determine if I should extend base functionality and use existing elements without having to deal with the issues previously mentioned with application scoping. I also want my developers thinking about how anything they’re developing affects the larger platform. I think that not scoping actually encourages and trains design and development discipline.

I’ll caveat all this by saying that I learned ServiceNow prior to application scoping, and I am comfortable managing development and updates in a single scope. If you’re not comfortable designing and developing with the whole platform in mind, by all means use the tools ServiceNow has provided. I’m not saying it’s a bad thing, I’m simply saying that I don’t think it’s necessary and can have negative side effects. As always, think before you act

Happy developing!

Over the years I’ve seen this flexibility used in a myriad of ways, some really clever and some head-scratchers. Most fall somewhere in the middle. It’s also been the impetus for the rise of good and best practices, from tribal hearsay to attempted codification. But one thing I feel strongly about is using the pieces of the platform in the best way possible for maintenance by non-coders. This is my good practice.

One of the elements that exists all over the platform is the ability to script your way to a solution. Often times it’s an “advanced” option on a particular element – think Business Rules and Transform Map scripts. For people with a coding background, there’s also ways to use object-oriented concepts to build classes and extensions and call them from most anywhere you need them. What we’ve seen in recent years is ServiceNow itself has moved into this for much of its new product development. Anywhere advanced logic is needed – something that requires logic that can’t be contained in a single core element – ServiceNow development is building advanced, decomposed scripts (usually in standalone Script Includes) and calling them from script blocks in platform elements: ACLs, Business Rules, UI Actions, etc. The issue with this is it requires a coding background to either understand the logic flow, or to make changes required for your business.

(I’m going to go on a slight rant here. If you don’t care to read my angst and just want my good practice suggestion, skip to the next paragraph.) In my opinion, this is simply poor design and is indicative of turning ServiceNow over to coders and not critical thinkers. The founders of ServiceNow gave us all the elements we needed to create code-light solutions but they aren’t being used. I often say to customers, “you can code your way to anything, but this isn’t good design”, but I also see a lot of customers whose admins or outside contractors working on their instances just write code until a problem is solved. Great. So now any troubleshooting or changes require going through mountains of code to try and solve. This is not a sustainable maintenance model and should concern anyone making budget decisions about paying for ServiceNow development and maintenance.

So what am I getting at with this article? What I’m suggesting is that if one is smart with designing solutions, these solutions can be maintained through configuration and not code. The solution looks something like this:

In this scenario, the UI Element is out of the box, and the code and config are created by the developer. What’s important to note is that the code is written once, as part of the development of the solution, and changes to the logic are done through the configuration. In other words, the code is simply a mechanism to connect the business logic to the front end, and the logic is contained in configuration.

Let me go through a more specific example of this. I’ve worked with quite a few customers who are using Customer Service Management (CSM) and ITSM and need them to work together. From a business logic standpoint, CSM Cases need to integrate to ITSM records (Incident, Change, Service Request) when non-CSM teams are needed to solve an issue or fulfill a request in order to complete a Case. The Case remains the purview of the customer and the CSM Agent teams, but backend teams and processes are needed to “finish the job”. I call this a Process Integration or intra-ServiceNow integration – it works much the same as a ServiceNow to other system integration (or SN instance A to SN instance B), but is managed wholly within a single instance and works to integrate disparate records and processes rather than systems.

ServiceNow has recognized this need and created a Service Management integration plugin (check name), but this solution is all code, 100%, down to the field level. If you need your process integration to work any differently than what ServiceNow has provided, you have to write your own code to make it work.

To me, this is clearly not ideal. So what I’ve done is something that looks like this:

The key is that we use mapping records to maintain what should go where, when.  The code simply is middleware between the maps and the application – it is kicked off when needed by the application, does lookups against the maps and updates records accordingly. The map records tells the system the following:

• What record and field to pull from
• What record and field to push to
• When the transaction should occur*
• Any field value translations required

* If you’re really slick, you use condition builders on the map records to determine the when. For example, you may only update the Case State to resolved when the integrated Request completes. Rather than building lots of Business Rules or writing If/Elses into your code, you build it into the mapping record. Now customer admins and even “admin-lites” can maintain business logic in configuration records and not code!

The elements described above can be extended and used for any Task integrations required across the platform. The steps are:

• Create Task Mappings as required
• Create Business Rules on the source tables – I usually just add a condition that the source record has to have a related Task-type record – that call the Script Include and pass the source and destination information to them.

I don’t consider any of this out of left field. Why? ServiceNow has already provided a model for this type of solution: Transform maps. While they were designed for importing data into the system, the use case is the same: We need to get data from one place to another, and we need allow for transformation of that data and logic to say when the data should be inserted, updated, or ignored. So why not use the same model for a process integration? 

I’ve described a specific use case, but if you’re thinking along with me, you realize this way of thinking can be used for other development scenarios. For example, I’ve seen quite a few consultants and customers (including myself) develop Service Request fulfillment lookup tables for Approvals, Tasking, routing and other data points. The goal is not to have to maintain dozens or hundreds of workflows (flows) or to have to checkout and edit a workflow every time a simple configuration change is needed. Ultimately, the lesson is this: when you’re designing a solution to a business problem in ServiceNow, always think, “Can I build this in a way that a non-coder can maintain the solution? Does it have to be solved with code only?” I suggest the answer to the latter is rarely “yes”.

Happy designing!

In recent years in the ServiceNow ecosystem, one of the topics that has taken on a “blob-like” existence is upgradeability: the ability to perform an upgrade to the next ServiceNow release, how long it takes, how much it costs, how much remediation is required, and to minimize all these aspects. I’d posit this is directly related to the increasing costs of the platform, and the ensuing exposure to C-level organizational people, and in particular, those who control the corporate purse strings. In conjunction with this, catchphrases such as “How to make your instance upgrade-proof” and “Stay out of the box (to make upgrades easier)” have taken on an inflated importance in companies’ ServiceNow journey. 

For this article, I’d like offer my opinions on the following topics, based on my experiences of 11 years of dealing with this:

• What actually affects your upgrades? What does it mean to take ownership of a ServiceNow element and go away from “out of the box”? What does it mean for your upgrades?
• How do you configure, design, develop and customize in ways that upgrade-proof your instance? 

How Upgrades Work

First, let’s talk about how an upgrade really works. ServiceNow releases a new version of the platform, which includes “black box” items and new items stored in the database. We don’t care about the first (and can’t even if we wanted to). The new database-stored items* are what cause us potential work. There’s three (3) scenarios when ServiceNow attempts to push these items with the upgrade:

1. A net new item is inserted into your instances. No conflicts occur because it is brand new, not seen in previous versions of the platform.
2. An existing item has a new version that updates your instance. You have never touched the item, so it upgrades with no issues.
3. An existing item has a new version that attempts to update your instance, but the existing item has been “touched” by you, so the upgrade skips the update, and you are left to research the differences and determine what to do. This is where the real upgrade work occurs.

* ”Items”: The ServiceNow configuration elements we are familiar with: Business Rules, UI Actions, Policies, Scripts, etc., that live inside of the ServiceNow database

So how does ServiceNow determine what should be skipped? If you’ve worked in ServiceNow for a while, you’re probably familiar with the “sys_update_xml” (“Customer Updates”) table. This is where every configuration or customization you’ve done is captured – the record of every Update Set entry. Without going into detail, any database table in ServiceNow that contains the metadata items that make up the system configuration is marked, and the system adds an entry to the sys_update_xml when any record in these tables is created, updated or deleted. If you update or delete a metadata item (record) that ServiceNow provided “out of the box”, it’s in sys_update_xml and you now “own” it. If you’re following the logic, the ServiceNow upgrade checks sys_update_xml when it attempts to update the metadata items. If it finds an entry in this table, it skips the update. (This includes marking a record inactive! I don’t know how the urban legend began that unchecking “Active” is upgrade-safe, but it’s not.)

I’ve include a screenshot representation of this here:

What this means is you’ve now got a skip log entry, and if you’re doing your due diligence, you check why and determine if and how to remediate. When something is skipped, it is advisable to review and understand what might have been lost in the skip. But it may not be required – “losing out” on new functionality doesn’t mean the system isn’t going to work! 

The other type of upgrade diligence is regression testing your current functionality to make sure nothing broke. This is particularly important for configuration changes and customizations you have made. When something breaks, it could be as a result of a skip. Usually this is a process break rather than an obvious UI issue or system error. For example, a new or change to a Business Rule is conflicting with a new Business Rule ServiceNow provided and causing logic issues within an Application lifecycle flow, rather than an ominous pop-up error message.

Upgrade Proofing Your Work

The previous section is very tactical in nature – what happened, what happens, and what to do about it. Now I’d like to discuss some ways of thinking about your work on the platform that might ease your upgrade processes. We’ll call it “Strategic Design for Easy Upgrades”. 

When you have to develop something on the platform, first consider the Business Requirements being presented. Think about what ServiceNow has provided Application-wise “Out of the Box”, and see how your company’s requirements align with them. Consider particularly:

• Does the lifecycle of the use case align with the out of box application? 
• Does the use case have special security requirements?
• Does the use case have special data schema requirements for a single entity (record)?
• Does the use case need special notifications?
• Do end users need the ability to save something as a “draft” before submitting?
• Do the reporting requirements align with the data schema?

When the business requirements don’t align with any out of the box application or function*:

• First consider an out of the box application with adjunct functionality
• Then consider a custom application
  * and the business requirement isn’t flexible

When I say “adjunct functionality”, what I mean is new functionality that runs in parallel with the “out of the box” functionality, not as a replacement or a customization to an existing item. For example, a brand new email notification with a different set of conditions and different content, or a new Business Rule that makes an update to a field that does not drive the application’s lifecycle flow.

I thought about some of the different development actions I’ve taken over the years and put them in the table below, categorized as “Start worrying” and “Sleep easy”, aka, “Probably shouldn’t do” and “OK to do”. In some cases there isn’t an equivalent in the “Start worrying” category, which I suppose is a good thing:

So to wrap it up, what I’m saying is if you follow the strategies from part 2 of this article, your work described in part 1 should be a lot easier. And, you don’t have to stay “Out of the Box” to do it. You simply have to understand what it means and work smartly with and around it to make your upgrades painless.

ServiceNow was designed as a platform, with a host of reusable elements. Using these for your business needs shouldn’t get you into upgrade trouble.

However… Just because you can do it, doesn’t mean you should!

Happy developing!

* – A play on the famous James Carville quote about the economy, not implying that ServiceNow folks are stupid

As many know, ServiceNow has been through 4 CEOs: Fred Luddy, Frank Slootman, John Donohoe, Bill McDermott; and as most know, Fred was the founder. Fred at his heart is a developer. (From Broke To Billionaire: How Fred Luddy Built The World’s Most Innovative Company) From Forbes:

Luddy hit on a revolutionary idea he had learned from three decades of programming: As the company’s name suggests, he’d deliver office services over the internet, on a subscription basis (monthly, per customer), updated easily without requiring customers to manually download software from disks on different operating systems. As this was 2003, Luddy proved to be a software-as-a-service pioneer, pushing a user-friendly interface designed for the average office worker…
The market’s response: “Meh.”

“We had this really great, simple platform for creating workflows, and we would go to people and say, Hey, you can do all these things with this, and they just weren’t interested,” recalls Luddy, who at one point sold a car to make payroll. “So we went back and said, Okay, we say this is this great tool for doing things like IT-support management, so why don’t we back that up and make an IT-support product?” This time the market bit.

I think there are two takeaways from this: ServiceNow was designed as a platform first, and the market only bit on the vision after Fred and team layered pre-canned applications (products) on top. The product just happened to be ITSM, because that’s what Fred knew from Peregrine, and the market was ripe for innovation in this space.

If you’ve seen Fred speak about his original vision for the platform, you know it came out of his desire to have a tool that could easily re-create form-based processes. Why do something on a paper form if it could easily be re-created in a computer based- system? So the platform was designed around this concept – the rapid development of form-based applications. This included the following elements:

• The ability to create new tables in the database
• A set of database field types that have inherent UI functionality (e.g. Reference and Journal fields)
• Inherent UI elements on top of (almost) any database table
• The ability to extend existing database table and inherit not only the data elements, but the UI and flow elements as well (as needed)
• The ability to create and run graphical workflows against (almost) any database table
• The ability to send and receive email notifications against (almost) any database table
• Graphical and script-based configuration elements that let you manage the UI to business process needs
• Server-side script-based business logic elements
• Graphical and script-based security configuration

The real key in this is the inherent elements that were “baked” into every table in the database, even those created by customer admins. If you create a new table, you automatically get default form and list views of that table’s data, and the ability to add or layer any of the other listed aspects to it. So customers could quickly spin-up new applications by creating tables and configuring how it is presented in form, lists and reports, and layer in business logic as desired.

Similarly, the original developers simply used these platform elements – that any ServiceNow platform administrator can access – to create a set of ITSM applications. In the 15+ years since, ServiceNow development has continued to use these elements to build other application sets: Customer Service, Human Resources, IT Operations Management, etc. But what hasn’t changed is the platform under these applications.

All of this is to say that the real power of ServiceNow is the platform, not the applications that sit on top of it. I realize that the average ServiceNow admin and/or developer doesn’t have the insight into the history, standards, business processes and other elements of every application and application suite in the platform. (For example, I have very little knowledge of all the reasons HR Management is built the way it’s built.) So there’s value to having knowledge and expertise behind the creation of a particular business application. However – and this is where the curmudgeon in me emerges – what has happened in recent years – really since Fred stepped back and stepped down from the company – is ServiceNow has tilted its marketing focus from being a platform company that happened to do ITSM, to being a product company that happens to run on a flexible platform. I doubt this shift was fully intentional or fully clean; rather, it’s been a gradual shift based on several factors:

1. Changes in company leadership: ServiceNow is now on its fourth CEO, along with the requisite management changes that happen along with it. Each change brings in leadership with their own histories and biases, without the knowledge of the platform’s history.
2. Responding to market feedback: I’ve heard from C-level ServiceNow folks that the platform direction is at least partially dictated by conversations had with customer C-level folks. These are conversations that rarely focus on underlying technology but rather broad business needs, particularly financial in nature. There’s a reason “stick to out of the box” became a thing, and this is a big part of it.
3. Aligning to shareholder needs: Going public means the company is driven by quarterly and annual financial results moreso than innovation. You can draw a straight line from the ServiceNow product releases of recent years – and their accompanying licensing – to quarterly financial goals that edify the markets. (For a great read on what this mentality does to companies, see Christensen’s “The Innovator’s Dilemma”, ISBN: 978-0062060242.) It’s easy to draw a straight line from Importance of Financials to Licensing to Focus on Product and not Platform. You can’t easily sell Platform improvements, but you can easily sell and market new Products.

You can’t necessarily argue with the results. (Check the share price.) My concern is that like a lot of successful companies that lose sight of what made them great, the turn from Platform to Product will catch up with them sooner or later. It’s a testament to how great the core platform is that the success continues, but as the complexity of the Products ServiceNow builds on top of it continues to increase, and admins and developers are hamstrung by the need to stay out of the box – rather than adopting the platform to their business needs – one wonders how much Platform goodwill will erode over time.

Last note: I’ve been saying for years now that it feels incongruous every time I attend a Knowledge conference and hear, “Upgrade-proof your instance – Stay out of the box!” in one ear and eye, and in the other, “CreatorCon! Create anything on ServiceNow!”. This incongruity goes hand in hand with what I’ve outlined in this article. Ultimately, I still love the platform but struggle to love its products.

(I’ll have more about “out of the box” and “upgrade proofing” in other articles.)