静态代码分析中可能存在的10大错误

Top 10 Mistakes with Static Analysis

All too often, teams flirt with static analysis for a few months or a year, but never truly commit to it for the long term. This is a shame because static analysis, when properly implemented, is a very powerful tool for eliminating defects—with minimal additional development effort.

At Parasoft, we've been helping software development organizations implement and optimize static analysis since 1996. By analyzing the good, the bad, and the ugly of static analysis deployments across a broad spectrum of industries, we've determined what mistakes are most likely to result in failed static analysis initiatives.

Here's what we've found to be the top 10 reasons why static analysis initiatives don’t deliver real value—and some tips for avoiding these common pitfalls.

10. Developers not included in process evolution

Don't overlook the developers when you're starting and fine-tuning the static analysis process. Since they're the people who will actually be working with static analysis on (hopefully) a daily basis, you'll get much better results by working with them from the start.

?When you're selecting a tool, get their gut reaction as to how easy it is to use and whether the tool fits reasonably well into their daily workflow. Any new practice that you

introduce will inevitably add some overhead to the workflow; the more you can minimize

this, the better.

?When you're working on the initial configuration (more on this in #7), be sure to get developer feedback on what kind of problems they're actually experiencing in the code.

You can then configure static analysis to help them identify and prevent these problems.

?On an on-going basis, check in with developers to see what rule violations seem noisy, incorrect, or insignificant to them. This feedback is helpful as you evolve and optimize the rule set. If a particular rule is generating noise or false positives, see if reconfiguring the

rule (e.g., by tweaking the rule parameters) might resolve the problem. If the developers

don't believe a certain rule is important, you can either try to convince them of its

significance (if you really think it's worth the fight), or you can stop checking it for the time being.

If you want to promote long-term adoption, you need to ensure that the static analysis is deployed in a way that developers recognize its value. Each time a violation is reported, you want them to think, "Ah, good thing the tool caught that for me" not "Ugh, another stupid message to get rid of." The more closely you work with developers, the better your chances of achieving this.

9. Unrealistic expectations

Some of the most common reasons for adopting static analysis are:

?Because everyone is talking about it

?To decrease costs

?To reduce development time

?To increase quality

Organizations that introduce static analysis because it seems like "the thing to do" understandably have a difficult time determining if static analysis is really worth it—and trying to convince team members to get on board with the initiative. Plus, without a clear goal, it's all too easy to make many of the other mistakes on this list. For instance, when teams aren't focused on preventing a specific category of defects, they are commonly guilty of enabling too many rules. And without a business driver, they commonly lack management buy in.

When the goal is to decrease costs and/or development time, it's important to realize that although this is feasible in the long term, introducing static analysis will actually increase costs and time in the short term. This is inevitable any time that you add a step to the development process. At first, you'll lose time as people learn how to run the tool and respond to the results. This can definitely be mitigated with automation, workflow, training, etc., but it cannot be eliminated. Later on, as developers become comfortable with the process and start cleaning their code, it will pay off in spades.

In terms of reducing development time and costs, it’s important to set your sights on the long term. It typically takes a few iterations with static analysis to see the gains you're hoping for: ?First iteration: Since you're just starting off and (hopefully) spending time on training,

this will probably be a negative time-wise, but a positive quality-wise.

?Second iteration: By now everyone will be more comfortable with static analysis and you won't be losing much time to training. There might be a zero-sum gain on time, and a little larger improvement on quality.

?Third iteration: At this point, you should start to see some pretty significant payback in terms of time as well as quality. By now, the process is baked in, people understand how to do it, a lot of the violations have been cleaned, you're starting to ramp up the rules,

you’re bringing more legacy code under compliance, and so on. This is where you start to reap significant rewards in terms of decreased development time and radically improved quality.

Try to be as specific as possible about your expectations. For instance, instead of aiming to "improve quality," strive for something more specific—like reducing the number of security breaches or field-reported bugs. This not only makes it easier to measure your progress, but also increases your chance of achieving your goal…provided that you use this specific goal to drive your static analysis initiative.

Start off by performing a root cause analysis to determine if you can really prevent the desired problems with static analysis—and if so—how you need to set it up to achieve this. When you focus the rules, configurations, policy, etc. on clear goals that make business sense, the initiative is more likely to meet your expectations.

8. Taking an audit approach

Sporadic audit scans tend to overwhelm developers, ultimately leaving the team with a long list of known problems, but little actual improvement. When a static analysis tool is used near the end of an application development cycle and it produces a significant amount of potential issues, you've

got a great report—but can you feasibly fix the code now? It's a lot like writing a large program in a new language—but failing to compile anything until every piece is completed.

A typical response is to then triage the results in order to determine which ones to fix and which to ignore. This is like trying to spell-check a document without having the proper dictionary—you waste a lot of time and miss important issues. In addition, now that you're aware of problems, proceeding without fixing them could open the door to charges of negligence in the unfortunate event that these dangerous constructs actually result in defects that cause real-world damages. The true value of static analysis comes from day-to-day incremental improvements in developers' coding habits and knowledge base—and audit-type approaches don't do much to foster this. It's designed to be a preventative strategy, not a QA tool. When teams run static analysis infrequently, they typically skim over a long list of results and cherry pick some items to be fixed. This eliminates some problems, but doesn't approach the level of quality improvement that a continuous approach could advance. Moreover, in a regulated environment, it also makes it considerably more difficult to convince auditors that your defined quality process is actually being followed in practice.

Another problem with the audit approach is that it tends to prioritize pretty reports over a practical workflow. Reports can be helpful—especially when you need to demonstrate regulatory compliance (e.g., for medical, military/aerospace, automotive, or other safety-critical software). However, if you ever need to choose between good report and a good workflow, definitely select the workflow. After all, if the workflow is operating properly, all the violations should be cleared before the code is checked in—so the reports will simply state that analysis was run and no issues were found.

7. Starting with too many rules

Some eager teams take the "big bang" approach to static analysis. With all the best intentions, they plan to invest considerable time and resources into carving out the penultimate static analysis implementation from the start—one that is so good, it will last them for years.

They assemble a team of their best developers. They read stacks of programming best practices books. They vow to examine all of their reported defects and review the rule descriptions for all of the rules that their selected vendor provides.

We've found that teams who take this approach have too many rules to start with and too few implemented later on. It's much better to start with a very small rule set, and as you come into compliance with it, phase in more rules.

Static analysis actually delivers better results if you don't bite off more than you can chew. When you perform static analysis, it's like you're having an experienced developer stand over the shoulder of an inexperienced developer and give him tips as he writes code. If the experienced developer is constantly harping on nitpicky issues in every few lines of code, the junior developer will soon become overwhelmed and start filtering out all advice—good and bad. However, if the experienced developer focuses on one or two issues that he knows are likely to cause serious problems, the junior developer is much more likely to remember what advice he was given, start writing better code, and actually appreciate receiving this kind of feedback.

It's the same for static analysis. Work incrementally—with an initial focus on truly critical issues—and you'll end up teaching your developers more and having them resent the process much less. Would you rather have a smaller set of rules that are followed, or a larger set that is not?

This might seem extreme, but we’ve found that it's not a bad idea to start with just one important rule that everyone follows. Then, once everyone is comfortable with the process and has seen it deliver some value, phase in additional rules.

Out of the hundreds or sometimes even thousands of rules that are available with many static analysis tools, how do you know where to start? We recommend a few simple guidelines:

1. Would team leaders stop shipping if a violation of this rule was found?

2. (In the beginning only) Does everyone agree that a violation of this rule should be fixed?

3. Are there too many violations from this rule?

6. Unwieldy workflow integration

Static analysis quickly becomes a hassle if your static analysis tool doesn't integrate into your development environment. For instance, assume you're trying to deploy a tool that delivers results via an email message. A developer who receives an email with a rule violation and a stack trace has to:

1. Find and open the related file in his development tool.

2. Locate the line(s) responsible for the reported problem.

3. Shift back and forth between the email and the editor to figure out what the message

means.

4. Go to some external reference to learn about what the rule checks, why it's important,

and how to fix a violation.

5. Manually fix the violation.

6. Wait for another automated scan to confirm that the violation was cleared.

This is so inefficient that it typically becomes an impediment to long-term adoption.

This was a fairly common practice about a decade ago, but it's since been replaced by more useful approaches—like Mylyn and other tools that inject results directly into the development environment. From the IDE, you can jump directly to the code responsible for the violation, review it, fix it, and check the updates in to source control. In many cases, you can even use a "Quick Fix" option to automatically refactor the code into compliance.

We recommend running desktop analysis on a daily basis, then using a server run to double check that nothing slipped through the desktop analysis. With this approach, make sure you have the same configuration on both the desktops and the sever. If the developers clean their code according to the desktop analysis, then still receive warnings from the server analysis, they're likely to question the value of performing desktop analysis.

You want to do anything you can to reduce the time required for static analysis—not just the time it takes to run the tool, but also the time involved in finding and fixing the violations. This means: ?Well-thought-out error messages

?Useful stack traces

? Low false positives

?Good rule descriptions that explain how to mitigate the problem

?Quick fixes that automatically refactor code into compliance

5. Lack of sufficient training

Some organizations claim that they don't see the need for static analysis training. Admittedly, static analysis is much simpler than other verification techniques. Nevertheless, it's important to train on how to:

?Install the tool

?Configure the tool with the appropriate rules

?Set up the build to perform static analysis

?Run the tool on the desktop

?Receive results from continuous integration / server runs

? Resolve violations

? Use suppressions

Granted, most of these issues don't warrant extensive instruction. However, teams that are reluctant to do even a brief "lunch and learn" on these issues typically end up with team members wasting time and thinking that static analysis is more of a hassle than it really needs to be.

It's a lot more effective to spend a little time upfront to get people started on the right foot than to throw it out there, see what problems surface, then try to overcome the resistance that has understandably developed.

4. No defined process

If you ask the team to perform static analysis without defining how it should be performed, the value is significantly diminished.

Before you start, it's important to sit down and think about the overall impact of static analysis—in terms of the developers, of course, but also for the build, the team as whole, the deployment, QA, etc.—and figure out the best way to integrate static analysis into your process. This job is often passed on to the build team. However, we recommend thinking twice before doing this. The build team will have great insight into how static analysis will impact the nightly build. Yet, what you really need is input on how it will impact developers and the overall process.

Since developers will be interacting with static analysis on a daily basis, it's best to cater to their concerns first and foremost—even if it comes at the expense of a little extra initial setup or configuration. Nevertheless, recognize that developers are not necessarily process experts. You'll dramatically increase your chances of success if you designate a process person to shoulder the responsibility of crafting a process that suits the needs and concerns of everyone involved.

We've seen organizations achieve considerable success by vetting a process in pilot projects. Basically, this involves defining an initial process, then "test driving" it with one group—preferably one actively working on important projects and willing to try new things. Make some adjustments to work out any initial kinks, then when it seems to be running smoothly here, roll it out to another group—ideally, one working in a very different manner or engaged in a dramatically different kind of project. Adjust as needed again, then deploy the optimized process across the organization. The advantage of this pilot approach are twofold:

?You don't subject as many people to the changes that are inevitable when you're optimizing the process.

?Since the process has been fine-tuned by the time of the main rollout, you'll be introducing a much more palatable process—thereby increasing your chance of success.

3. No automated process enforcement

Without automated process enforcement, developers are likely to perform static analysis sporadically and inconsistently. The more you can automate the tedious static analysis process, the less it will burden developers and distract them from the more challenging tasks they truly

enjoy. Plus, the added automation will help you achieve consistent results across the team and organization.

Many organizations follow a multi-level automated process. Each day, as the developer works on code in the IDE, he or she can run analysis on demand—or configure an automated analysis to run continuously in the background (like spell check does). Developers clean these violations before adding new or modified code to source control.

Then, a server-based process double checks that the checked in code base is clean. This analysis can run as part of continuous integration, on a nightly basis, etc. to make sure nothing slipped through the cracks.

Assuming that you have a policy requiring that all violations from the designated rule set are cleaned before check in, any violations reported at this level indicate that the policy is not being followed. If this occurs, don't just have the developers fix the reported problems. Take the extra step to figure out where the process is breaking down, and how you can fix it (e.g., by fine-tuning the rule set, enabling the use of suppressions, etc.).

2. Lack of a clear policy

It's common for organizations to overlook policy because they think that simply making the tool available is sufficient. It's not. Even though static analysis (done properly) will save developers time in the long run, they're not going to be attracted to the extra work it adds upfront. If you really want to ensure that static analysis is performed as you expect—even when the team's in crunch mode, scrambling to just take care of the essentials—policy is key.

Every team has a policy, whether or not it's formally defined. You might as well codify the process and make it official. After all, it's a lot easier to identify and diagnose problems with a formalized policy than an unwritten one.

Ideally, you want your policy to have a direct correlation to the problems you're currently experiencing (and/or committed to preventing). This way, there's a good rationale behind both the general policy and the specific ways that it's implemented.

With these goals in mind, the policy should clarify:

?What teams need to perform static analysis

?What projects require static analysis

?What rules are required

?What degree of compliance is required

?When suppressions are allowed

?When violations in legacy code need to be fixed

?Whether you ship code with static analysis violations

1. Lack of management buy-in

Management buy in is so critical to so many aspects of static analysis success that you simply can't get by without it. Think about it…

?Policy—set by management

?Process—defined by management

?The configuration, the business case—driven by management

On the one hand, management has to be willing to draw a line in the sand and ensure that static analysis becomes a non-negotiable part of the daily workflow. There has to be a policy for how to apply it, and that policy has to be enforced.

On the other hand, management has to understand that requiring static analysis has a cost, and ensure that steps are taken to account for and mitigate those costs. Mandating compliance to a certain set of rules without adjusting deadlines to account for the extra time needed to learn the tool (plus find and fix violations) is a recipe for disaster.

The most successful static analysis adoptions that we've seen are all backed by a management team that knows what they want static analysis to achieve, and is willing to incur some costs in the short term in order to achieve that goal in the long term.

The beauty of having the whole process set up well is that if it's not working as you expect, it's easy to analyze, understand, and correct. But if you lack management buy in, you probably won't have compliance with the process—and it's hard to determine whether there are fundamental weaknesses in the current process that need to be resolved.

Closing Thoughts: Comprehensive Development Testing

It's important to remember that static analysis is not a silver bullet. You can't rest assured that a component functions correctly and reliably unless you actually exercise it with test cases. Even the best implementation of static analysis cannot provide the level of defect prevention you could achieve through consistent application of a broad set of complementary defect

detection/prevention practices—in the context of an overarching standardized process. Parasoft's Development Testing platform helps organizations achieve this by establishing an efficient and automated process for comprehensive Development Testing:

?Consistently apply a broad set of complementary Development Testing practices—static analysis, unit testing, peer code review, coverage analysis, runtime error detection, etc.

?Accurately and objectively measure productivity and application quality

?Drive the development process in the context of business expectations—for what needs to be developed as well as how it should be developed

?Gain real-time visibility into how the software is being developed and whether it is satisfying expectations

?Reduce costs and risks across the entire SDLC

Next Steps

To see specific examples of how leading organizations achieved real results with static analysis, visit Parasoft's Static Analysis Resource Library. For example, you can learn how Parasoft's static analysis helped:

?Samsung – Accelerate development while maintaining stringent quality standards.

?Cisco – Comply with corporate quality & security initiatives without impeding productivity.

?Wipro – Achieve strict quality objectives while reducing testing time and effort by 25%.

?NEC – Streamline internal quality processes to more efficiently satisfy quality initiatives.

About Parasoft

For 25 years, Parasoft has researched and developed software solutions that help organizations deliver defect-free software efficiently. By integrating development testing,

API/cloud/SOA/composite app testing, dev/test environment management, and software development management, we reduce the time, effort, and cost of delivering secure, reliable, and compliant software. Parasoft's enterprise and embedded development solutions are the industry's most comprehensive—including static analysis, unit testing with requirements traceability, functional & load testing, service virtualization, and more. The majority of Fortune 500 companies rely on Parasoft in order to produce top-quality software consistently and efficiently. Contacting Parasoft

USA

101 E. Huntington Drive, 2nd Floor

Monrovia, CA 91016

Toll Free: (888) 305-0041

Tel: (626) 305-0041

Fax: (626) 305-3036

Email: info@https://www.360docs.net/doc/a73838666.html,

URL: https://www.360docs.net/doc/a73838666.html,

Europe

France: Tel: +33 (1) 64 89 26 00

UK: Tel: + 44 (0)208 263 6005

Germany: Tel: +49 731 880309-0

Email: info-europe@https://www.360docs.net/doc/a73838666.html,

Other Locations

See https://www.360docs.net/doc/a73838666.html,/contacts

? 2012 Parasoft Corporation

All rights reserved. Parasoft and all Parasoft products and services listed within are trademarks or registered trademarks of Parasoft Corporation. All other products, services, and companies are trademarks, registered trademarks, or servicemarks of their respective holders in the US and/or other countries.