“Thinking about design is hard, but not thinking about it can be disastrous.” – Ralph Caplan

Introduction

For almost 14 years in the ServiceNow space, and across a rapid expansion of the exosystem, it has been interesting to observe and analyze various organization’s approaches to developing and maintaining their ServiceNow environment. Specifically, how do organizations manage the inflow of business needs, the distribution and velocity of development, configuration and administrative work, and the ongoing maintenance of the platform? As the footprint of ServiceNow has expanded conjunctionally with the expansion of the business functions the platform supports, I increasingly see divergence in these strategies. I’m writing this article to articulate these strategies and provide my view of how each succeeds and fails, along with my recommendations for the correct strategy given a company’s view of ServiceNow.

Management Approaches

There really are two ends of the spectrum for how ServiceNow environments are managed. At one end is the idealized view and what ServiceNow itself espouses: Use of Idea and Demand Management to receive and vet business requirements, an oversight board – which ServiceNow calls a “Center of Excellence” – who does the vetting and prioritization of these requirements, Agile and PPM to manage the development work to fulfill these requirements, and an operational organization that handles release management, break/fix work, upgrades, and performance and security of the platform. If you’ve got your thinking cap on while reading this, you’ll quickly sense that this is intended and works best in the largest ServiceNow implementations, where the platform is a large part of an overall enterprise strategy for a company.

The other end of the spectrum stems largely from traditional IT functions; that is, a purely operational model and mindset where all development is treated as one-off break/fix/enhance type work. Where “keep the lights on” is the primary and sometimes only strategy. In these organizations, ServiceNow development work is typically handled through Incident and/or Enhancement processes, and each task is designed, developed and released “in a silo”, usually without thought to larger strategic initiatives. In other words, the view of the development does not extend beyond the scope of the need elucidated.

With a 25 year career in IT, I’m certainly aware of and sympathetic to this mindset. I find it particularly prevalent in MSP or MSP-like organizations. It’s not that the people running these organizations intend to be “unstrategic” (not a word), it’s what they know. These mindsets are built over years and decades of running IT as an operational entity.

There is a cost to doing business this way – and this is the crux of this article. When you implement under an operational mindset, you necessarily build everything as one-off. Critically, there are no design or architecture considerations taken into account, which means there are concerns for platform maintenance, stability, health and optimization. These can range from the simplest quirks like inconsistent forms and fields, and re-creations of code logic, to large-scale issues with performance and user experience.

Examples

Here are some specific examples of development done without design or architecture prior to “hands-on” work:

• A developer customizes an out of box ServiceNow application when a custom application would have served the requirement better. This leads to upgrade issues.
• A developer builds a security requirement using client-side functionality, which is pseudo-security. This security hole is exposed when using integrations and server-side functionality to the same data.
• A series of requirements for a single application are developed as one-offs. After these are implemented, the UI/UX experience is compromised, as now the form design is cluttered and out of sync with other applications. Adding UI logic and many related lists hinders the performance of form loads.
• One developer uses Workflow, another Flow Designer, another a series of Business Rules, and another a series of Glide Ajax client scripts, all to implement similar requirements. Maintenance becomes hyper complex as each change or fix is a one-off “head scratcher”; “Now where is this functionality??”

I can argue that Agile is a contributor to this problem. Not the methodology itself, but the incomplete usage of the methodology. I often see organizations going no further than Stories to manage their work. While Stories done correctly are ideal for “state the business requirement, not the technical how” of documentation, without using Epics to group requirements into a cohesive release, and more importantly, without architectural design overseeing the Epic, the Stories are developed in silos and lead to the issues noted above.

Best Practice

In my experience, the best practice is to have an architectural or design review of all requirements prior to development. Some may only need a cursory review to confirm it can be built as a one-off. Others may need a directed technical approach because there are several ways it could be built, and a consistent approach platform-wide is best for long term maintainability. And some may need a complete analysis of pros and cons, buy versus build, custom application versus out-of-box in order to build the right solution for the business need and the platform sustainability.

I’ve included a diagram below that shows the “idealized” process flow, including a Center of Excellence that fulfills this function:

The concept of a Center of Excellence, or at least some level of architectural oversight, is not meant to be onerous or a process bottleneck. This is a common concern organizations have, and a reason they don’t do it. My argument is that the operational sweet spot for such a function lies somewhere in the middle of the spectrum: Organizations are free to be as fast and independent with business requirements as they choose. The oversight part of the process is a simple architectural design review of all Stories (requirements) prior to development, with the end result a proper technical approach to development. A good architect will quickly recognize when there are multiple approaches to implement a requirement and provide guidance on the correct approach, taking into consideration all aspects mentioned previously. If the Agile methodology is being used, this can be part of the grooming process.

The diagram above is one I drew for where the Center of Excellence lives in the overall development process, between the requirements gathering, and the execution, either as operational one-offs or as larger project-type initiatives.

ServiceNow’s Documentation on Center of Excellence

In the end, it comes down to an organizational decision, even if not made consciously: “Do we spend the time up front to ensure things are developed in a cohesive platform strategy way, or do we dedicate more time, money and resources to fixing issues when they (inevitably) rise in Production?”  The simple analogy is working to prevent forest fires, or dedicating resources to fighting forest fires.

“Continuous improvement is not about the things you do well — that’s work. Continuous improvement is about removing the things that get in the way of your work. The headaches, the things that slow you down, that’s what continuous improvement is all about.” – Bruce Hamilton

I’ve written previously about the power of the platform, and my belief in its terrific original design and flexibility. In recent years, in its push to create and sell products, ServiceNow has sacrificed enhancements to the platform which us architects, developers and admins have to work around and explain to our customers. In this article, I’ll discuss some of the enhancements I wish ServiceNow would implement now (and in some cases should have done long ago). While selfishly these would make my life and the lives of people who manage and work on ServiceNow easier, these are also features that will keep ServiceNow ahead of, or at least apace of, the competition. And let’s not overstate our selfishness – some of these are great for requesters and customers too.

Rich-text / HTML Comments and Work Notes

We don’t live in a plain text world any more. … ServiceNow should enhance Comments and Word notes fields to support rich text and HTML formatting. One of the important outcomes of having this feature is the ability to include inline images and marked up text, so that agents and customers can exchange examples in order to resolve issues more expediently. Consider any IT firm who is troubleshooting a customer issue via Customer Service Management. The ability for both the customer and the support agent to supply screenshots with text and arrows to explain the exact issue or fix is far easier to communicate and comprehend than a plain text explanation (“a picture is worth 1000 words”), or a text description of an attachment that don’t live together in the UI. This method is cumbersome and unintuitive. Additionally, the rich text / HTML notes can go out and be received via email – the bane of our existence but fundamental part of how “business is done” no matter how much we fight it or come up with alternatives. (I don’t know of any email system that doesn’t support this formatting.) Regardless of whether the customer is viewing these marked up notes in the Service Portal or via email, their experience is enhanced, and in best cases, their issue can be resolved faster.

Editable Comments and Work Notes

I’ll include this as a sub-header under the enhanced notes banner. If we’re going to make comments and work notes rich, let’s go ahead and make them editable after saving in select cases and to select people. I say the latter part because if an agent has entered an Additional comment and the system has informed the customer of the comment via email, it’s likely a bad idea to turn around and edit that note. However, there are plenty of cases where the ability to edit a Work note is useful, and sometimes security reasons why (someone has put a password in plain text in a note). I haven’t devised a hard and fast rule of what should be editable when and by whom; let’s start with the functionality and figure it out from there.

Enhanced Attachments

Attachment functionality has been basic since the platform originated: Users with write access to any record in the system can attach files to that record and users with read access can view those files. There isn’t much functionality beyond this other than the ability to add all the attachments to all instances of an outbound email notification.

Customers have been asking for years for additional functionality around attachments:

• Classifying attachments as internal (fulfillers or employees) and external (requesters or customers), much like Comments and Work notes
• Specifying more complex security around each attachment on a record
• Choosing particular attachment(s) to send with an email notification (in real-time)

I’m sure there are others but the point is made. Getting into the weeds on how attachments are stored in ServiceNow is a discussion beyond this article. Suffice it to say there is a great demand for greater flexibility around the classification and security of attachments, beyond “this attachment belongs to this record”.

Requested by and for at the Task* Level

*Assumes you understand the Task table hierarchy and inheritance.

This is one of the most common customizations implementers have been doing as long as I’ve worked on ServiceNow. The basic thesis is this: For every Task (every Task), the system should be able to record, track and report on who requested the work, and who it’s requested for. This seems so simple. I think the lack of this in the platform is residue from the early days when ServiceNow was primarily just an ITSM system, and as such, they put a Caller field on Incident, a “Requested by” on Change, a “Requested for” on Service Request, and then didn’t think past it. In subsequent applications, they added “Requested for” or “Requested by” on certain applications, but it’s not consistent across the platform.

(Some may say “what about the ‘Opened by’ field on Task?”. While it’s great that this field exists at the Task level, consider this: An administrative assistant calls in a request to the Service Desk for something for his CEO boss. The Service Desk opens the Request. In this case, the “Requested by” is the admin assistant, the “Requested for” is the CEO, and the “Opened by” is the Service Desk agent. I think this field is needed and serves a distinct purpose to the others.)

To this end, I’ve worked on many implementations and have often recommended these fields be added to the Task table and they are used as the defacto values on all Task forms, lists, reports, etc. In addition to having a consistent approach and data/field structure on all Tasks (work being performed), it also enhances reporting at the Task level, and can be used to report on organizational performance: How many requests is IT delivering to HR? And vice versa? Having your requesters and assignees all at the Task level, along with good core data, allows you to take your Service Management to the next level. But this should not fall on implementers to customize; ServiceNow should fix the platform so it’s “out of the box” this way.

More Granular Log Timestamps

This is a feature purely for admins and developers. Because the platform timestamps on all records in the Created (sys_created_on) and Updated (sys_updated_on) are granular only down to the second, it’s often hard to troubleshoot the order of processing execution. After all, many of these executions are happening at the millisecond level. For example, if you’re troubleshooting a complex script with lots of logging, when you view the Script Logs or the more general Platform Logs, because you can only sort down to the second, you can’t see exactly the order of your logging. Of course you can number your log statements, but you lose the order of other logging that may be occurring outside of your explicit statements. This is important when other things in the system may be impacting your code. In an ideal world, at least for logging, you could see the exact order of execution. Indeed, this was possible when I was working at ServiceNow and could elevate to maint access on the platform (access above admin only available to ServiceNow employees), and I can tell you from experience it made my troubleshooting much easier.

I’ll hedge my statements by saying this is only really necessary for logging – Task-based work and other auditing is typically fine at the hours:minutes:seconds level.

Other Quick Hits

Here are a few others that I’ve addressed in other articles or will be in the future:

• Complete Catalog Variable redesign (aka “Variables are the worst”)
• Upgrade Workflows or fix Flow designer to have a GUI approach like Workflow
• Stop creating so many scopes – how about 1 per application?
• Fix the out of box State model once and for all
• Create out of the box solutions that are less code-heavy, aka Drink your own “low code” Kool-aid. None of us want to try and troubleshoot by looking through dozens of undocumented function calls!

Conclusion

I’ve written about some of the most common areas of concern for customers, things I’ve learned from 13 years of ServiceNow implementation. There’s still plenty of power in the platform – it’s why many of us started working with ServiceNow and what keeps us evangelizing about its power. The ask is simple: ServiceNow should solicit feedback from its most experienced implementers, honing in on the most common platform concerns that birth customizations of all shapes and sizes, and devote some of their massive development resources to these changes and enhancements. I’m sure this can be done in parallel with licensed application development. So do it, and keep this platform great!

In journalism, there’s the concept of the Five W questions whose answers are fundamental to getting the information needed:

• Who
• What
• When
• Where
• Why

I want to talk about what I call the “Three Ws of Development” in the ServiceNow realm. These three are: When, Where and Why. We’re going to skip the questions “Who” and “What”. Why? Because “who” is a question for hiring managers, recruiting, and resource vetting. And “what” is (too often) the focus of most if not all training and documentation. Do you need to get the current user’s ID on the client side? Check the API – that’s the “what”. Instead, I want to focus on some areas of development consideration that I feel are often neglected, and I’ll explain each and try to put them in context.

Most everyone in the ServiceNow world knows the basic system architecture, which is the same for almost all cloud-based applications:

On the ServiceNow side, there’s an Application Server that stores the compiled code, the files (images, etc.) needed by the application, and delivers content to the user’s browser when requested. The App Server connects to a SQL Database that stores all the data records, and in ServiceNow’s case, all the configurations and code customizations created by both ServiceNow and customers.

Now consider a “normal” transaction between these entities. We’ll use one that’s fundamental to ServiceNow: Accessing and updating an Incident record. The following shows all the component parts in this transaction:

1. Client makes a call to the server to get a specific Incident record
2. Query Business Rules determine if certain records should be restricted from view
3. Application queries the database for that record from the incident and related tables
4. Database returns required data to the application server
5. ACLs are evaluated and applied
6. Display Business Rules run logic and place data on the scratchpad (to use client-side)
7. Application Server provides browser with form, data, and all configurations based on 4-6

8. onLoad UI Policies and Client scripts run
9. User manipulates data, onChange UI Policies and Client scripts run
10. UI Actions run client-side, server-side, and sometimes both (first client, then server)
11. On Save, client sends form data to the Application Server
12. Before Business Rules manipulate record data prior to saving to the Database
13. Application Server sends data to the Database
14. After Business Rules run on any table (current, previous still available)

This is a broader “order of execution” list than ServiceNow provides in their documentation, which deals strictly with Business rules.

So how does this apply to our Ws discussion? Let’s discuss:

Where

In considering where to develop your configurations and customizations, you will almost always get better performance having them run on the Application Server rather than the client’s browser session. Observe the middle section of the diagrams above and the components that live in it. For example, when a user accesses a record, if you need to run logic and have the result available on in the client’s session, it is faster performance-wise to run the logic in a display Business Rule and place the result in the scratchpad rather than run the logic in a Client script, particularly if the latter would require querying the database after the record has been accessed and loaded in the client browser session.

Another important use case is security. All security should be done in ACLs, and only supplemented with UI Policies and scripts where needed. ACLs are the only true security, all other methods are posing as security and can usually be bypassed. For example, let’s say you have a Social Security Number field on a User record that should only be available (visible) to Human Resources, and should not be editable by anyone (it feeds from an HR/HCM system). This field should be secured with field-level read ACL for users with an HR role, another field-level read ACL marked false for all other users, and field-level write ACL marked false for all users. If you were to use a data dictionary level read-only marker, this could be bypassed by scripts running as the system. If you were to use a UI Policy or a Client Script to make it visible and/or read-only, this could be bypassed by list views and edits, server-side scripts, and Integration APIs.

In keeping with the last idea, it is also good practice to mimic your client-side logic on the server-side. For example, if you don’t want to force a field to be mandatory for every save, but want to run client side logic that prevents the save of the record without the mandatory field in a particular scenario, you should also create a server-side Business Rule that aborts the save and messages the user about the mandatory field. This way, your logic is enforced in list edits and server-side record manipulation, and not just in form views.

• List edits bypass “standard” Client Scripts and UI Policies
• External systems integrate directly with the Application Server

Note: You may have noticed that I haven’t mentioned List configurations like List Edit client scripts. These are also usable to “fix” the List edit issues mentioned, but it doesn’t fix server-side logic and integrations.

When

Going hand-in-hand with the Where is the When of development. Specifically, consideration should be paid to when in the lifecycle of the full transaction the development is best to exist. Consider the following Business Rule scenarios:

• You need to access information client-side that is not available on the current record being accessed but can be ascertained using the current record (e.g. querying information from another table using field data from the current record. The correct time (when) to run this logic is in a display Business Rule and place the needed information in the scratchpad.
• You need to manipulate field data for the current record before the record is saved to the database; for example, you need to multiply two integer fields from the current record and store the product in a total field on the same record. The correct time (when) to execute this is in a before Business Rule.
• You need to manipulate field data for a different record using the current record data, and either need to access the “previous” object, or you need the manipulation to be reflected in the user’s updated client session view. For example, you are updating a related record to the current record and the update needs to be reflected in a related list of the record form the user is currently viewing. The correct time (when) to execute this is in an after Business Rule.
• You need to manipulate field data for a different record using the current record data but it doesn’t need to be reflected in the user’s updated client session view, OR you need to trigger an integration whose updates don’t need to be reflected in the user’s updated client session view. The correct time (when) to execute this is in an async Business Rule. Async Business Rules create a Schedule Job that gets queued and executed as system resources become available.

Why

There are many other scenarios an experienced system admin/developer can think of. A key is to understand all the component parts of the transaction diagrams above; the goal is to configure and develop in the correct place, both where and when. Why? The benefits of developing in the correct time and place are:

• Performance: Following the when and where guidance will ensure the best system performance. Running logic on the server is almost always faster than on the client. Running logic asynchronously means the client, and therefore the user, means they don’t have to wait for the transaction to complete. Each time the transaction can avoid another database query means faster execution and less wait time.
• Security: Developing measures that secure fields and records, both visibility and editability (R and U in CRUD), in the correct place, means your instance is actually secure, versus appearing to be secure to an end user. After all, most if not all security breaches do not come from a human clicking and typing.
• Maintainability: Broader maintainability comes from following good and best practices. Consider a ServiceNow health check – what is the likely result of a system audit if you follow the practices suggested above versus using whatever means are available to create solutions.

Conclusion

ServiceNow provides many means to an end. Sadly, much of the documentation and training does not cover which the correct means to the end; rather, it simply details, “If you need to do X, go here and do this”. It doesn’t say why you should do it this way, or what the overall implications are to doing it that way. What I’ve tried to do is give you a baseline of how the system works, so you can understand where and when the correct places are to do X. Understand the baseline, and stop and think about it before everything you develop. I promise your platform, and hopefully your work life, will be better for doing so.

Happy developing!