Chapter 1: IntroductionAbout This BookMicrosoft Excel is much, much more than just a spreadsheet. Since the introduction of the Visual Basic Editor in Excel 97 and the improved stability of Excel 2000, it has become a respected development platform in its own right. Applications written using Excel are now often found alongside those written using Visual Basic, C++, Java, .NET, etc, as part of many corporations' core suite of business-critical applications. Indeed, Excel is often used for the client end of web-based applications, made particularly easy with Excel 2003's XML import/export features. Unfortunately, Excel is still all too often thought of as a hobbyist platform; that people develop spreadsheet-based applications in their spare time to help out with their day job. A brief look at the shelves of any bookstore seems to confirm that opinion. While there are myriad titles explaining how to use Excel and numerous titles about Excel and VBA, there are none that provide an overall explanation of how to develop professional-quality Excel-based applications. This is that book. While all the other major languages seem to have a de-facto standard text that explains the commonly-agreed best practices for architecting, designing and developing applications in that language, Excel does not. This book aims to fill that gap. All three authors are professional Excel developers who run our own companies developing Excel-based applications for clients ranging from individuals to the largest multinational corporations. This book details the approaches we use when designing, developing, distributing and supporting the applications we write for our clients. This is not a beginner-level book. We assume that the reader will have read and (mostly) understood our Excel 2000/2002 VBA Programmer's Reference, John Walkenbach's Excel N Power Programming or similar titles. The Excel DeveloperExcel developers can be divided into five general categories, based on their experience and knowledge of Excel and VBA. To varying degrees, this book has something to offer each of them, but with a focus on the more advanced topics. Putting yourself into one of these categories might help you decide whether this is the right book for you. The basic Excel User probably doesn't think of themselves as a developer at all. To them, Excel is no more than a tool to help them get on with their job. They start off using Excel worksheets as a handy place to store lists or perform simple repetitive calculations. As they discover more of Excel's functionality, their workbooks become more complex and start to include lots of worksheet functions, pivot tables and charts. There is little in this book for these people, though Chapter 4 – Worksheet Design details the best practices to use when designing and laying out a worksheet for data entry, Chapter 14 – Data Manipulation Techniques explains how to structure a worksheet and which functions and features to use to manipulate their lists and Chapter 15 - Advanced Charting Techniques explains how to get the most from Excel's chart engine. Using the techniques suggested in these chapters should help the basic Excel user avoid some of the pitfalls often encountered as their experience and the complexity of their spreadsheets increases. The Excel Power User has a wide understanding of Excel's functionality, knows which tool or function is best to use in a given situation, creates complex spreadsheets for their own use and is often called on to help develop their colleagues' spreadsheets or to identify why their colleagues' spreadsheets don't work as intended. Occasionally the Power Users includes small snippets of VBA they found on the internet or created using the macro recorder, but struggles to adapt the code to their needs. As a result, they produce code that is untidy, slow and hard to maintain. While this book is not a VBA tutorial, the Power User has much to gain from following the best practices we suggest for both worksheets and code modules. Most of the chapters in the book will be relevant to the Power User who has an interest in improving their Excel and VBA development skills. The VBA Developer makes extensive use of VBA code in their workbooks – often too much. They are typically either Power Users who have started to learn VBA too early or Visual Basic 6 developers that have switched to Excel VBA development. While they may be very proficient at VBA, they believe every problem must have a VBA solution and lack sufficient knowledge of Excel to make the best use of its features. Their solutions are often cumbersome, slow and make poor use of Excel's object model. This book has much to offer the VBA Developer to improve their use of Excel itself, including explaining how to architect Excel-based applications, the best practices for designing worksheets and how to use Excel's features for their data entry, analysis and presentation. The book also seeks to improve their Excel VBA development skills by introducing advanced coding techniques, detailing VBA best practices and explaining how to improve their code's performance. The Excel Developer has realized the most efficient and maintainable applications are those which make the most of Excel's own functionality, augmented by VBA when appropriate. They are confident developing Excel-based applications for their colleagues to use or as part of an in-house development team. While their undoubted knowledge of Excel is put to good use in their applications, it also constrains their designs and they are reluctant to use other languages and applications to augment their Excel solutions. They have probably read John Walkenbach's Excel 97/2000/2002/2003 Power Programming and/or our own Excel 2000/2002 VBA Programmer's Reference and need a book to take them to the highest level of Excel application development – that of the professional developer. This is that book. The Professional Excel Developer designs and develops Excel-based applications and utilities for their clients or employer that are robust, fast, easy to use, maintainable and secure. While Excel forms the core of their solutions, they include any other applications and languages that are appropriate, such as third-party ActiveX controls, automating other applications, using Windows API calls, using ADO to connect to external databases, C/C++ for fast custom worksheet functions, VB6 or VB.Net for creating their own object models and securing their code and XML for sharing data over the internet. This book teaches all those skills. If you are already a Professional Excel Developer, you will know learning never stops and will appreciate the knowledge and best practices presented in this book by three of your peers. Excel as an Application Development PlatformIf we look at Excel as a development platform and not just a spreadsheet, we can break it down into five fundamental components we can use for our applications: The worksheet, charts etc, used as a user interface and presentation layer for data entry and reporting
The Excel object model, allowing programmatic control of (nearly) all of Excel's functionality, from both within Excel and from outside it. The Worksheet as a Presentation Layer for Data Entry and ReportingWhen most people think about Excel, they think in terms of typing numbers into cells, having some calculations update and seeing a result displayed in a different cell or on a chart. Without necessarily thinking in such terms, they are using the worksheet as a user interface for their data entry and reporting and are generally comfortable with these tasks. The in-cell editing, validation and formatting features built in to Excel provide an extremely rich and compelling data-entry experience, while the charting, cell formatting and drawing tools provide a presentation-quality reporting mechanism. It is hard to imagine the code that would be required if we tried to reproduce the experience using the form design tools available in most other development environments, yet it's there waiting for us to use in our Excel-based applications. The biggest problem we face is how to add some structure to the free-form grid of the worksheet, in order to present a simple and easy to use interface, while leveraging the rich functionality Excel provides. Chapter 4 – Worksheet Design introduces some techniques and best practices for developing worksheet-based data entry forms, while Chapter 15 - Advanced Charting Techniques discusses using charting capabilities. The Worksheet as a Simple Data StoreWhat is a worksheet when it's never intended to be shown to the end user? At its simplest, it's no more than a large grid of cells in which we can store just about anything we want to – numbers, text, lists, tables or pictures. Most applications use some amount of static data or textual or graphical resources; storing that information in a worksheet makes it both extremely easy to access using VBA and simple to maintain. Lists and tables in worksheets can directly feed Excel's data validation (as shown in Chapter 4 – Worksheet Design), greatly simplify the creation and maintenance of command bars (Chapter 8 – Advanced Command Bar Handling) and allow us to construct dynamic userforms (Chapter 10 – Userform Design and Best Practices). VBA – Excel's Programming LanguageWe expect most readers of this book will have at least some familiarity with VBA. If not, we suggest you read either our Excel 2000/2002 VBA Programmer's Reference or John Walkenbach's Excel 97/2000/2002/2003 Power Programming before continuing much further. Many people see the 'A' in VBA as meaning the language is somehow less than Visual Basic itself. In fact, both VB6 and Office 2000 and above use exactly the same DLL to provide the keyword, syntax and statements we program with. The only differences are the objects provided by the run-times (the VB runtime vs the Excel objects), the forms packages (VB's 'Ruby' forms vs Office UserForms) and that VB6 includes a compiler to create EXEs and DLLs, while VBA is always interpreted at run-time. Indeed, the Office Developer Edition (pre-Excel 2003) includes the same compiler VB6 uses, allowing us to compile (simple) DLLs from within the Office Visual Basic Editor. Most beginner and intermediate VBA developers use VBA as a purely procedural language, with nearly all their code residing in standard modules. VBA also allows us to create applications using an Object Oriented Programming (OOP) approach, in which class modules are used to create our own objects. Chapter 7 – Using Class Modules to Create Objects and Chapter 11 – Interfaces explain how to use VBA in this manner, while basic OOP concepts (such as encapsulation) are used throughout the book. Most of this book is dedicated to explaining advanced VBA techniques and a professional approach to application design and development which can put using VBA in Excel on a par with, and sometimes in front of, using VB6 or VB.Net for application development. We also show in Chapter 20 – Combining Excel and Visual Basic 6 and Chapter 22 – Using VB.Net and VSTO that the Excel developer can use the best of both worlds, by combining Excel, VB6 and/or VB.Net in a seamless application. The Worksheet as a Declarative Programming LanguageTake the following code: dSales = 1000 dPrice = 10.99 dRevenue = dSales * dPrice That could quite easily be a few lines of VBA. We give the variable dSales a value of 1000, the variable dPrice a value of 10.99, then calculate the revenue as sales times price. If we change the names of the variables and adjust the spacing, the same code could also be written as: D1 =1000 D2 =10.99 D3 =D1*D2 which looks much more like worksheet cell addresses and formulas than lines of VBA code, showing that a worksheet is in fact a programming language of its own, if we choose to think of it in those terms. The IF() worksheet function is directly equivalent to the If...Then...Else VBA statement, while the judicious use of circular references and iteration can be equivalent to either the For...Next or Do...Loop structures. Instead of stating a set of operations that are executed line-by-line, we 'program' in this language by stating a set of declarations (by typing formulae and values into worksheet cells), in any order we want to: "D3 is the product of D1 and D2" To 'run' this program, Excel first examines all the declarations and builds a 'precedence tree' to identify which cells depend on the results of which other cells and thereby determine the most efficient order in which the cells must be calculated. The same precedence tree is also used to identify the minimum set of calculations that must be performed whenever the value in a cell is changed. The result is a calculation engine that is vastly more efficient than an equivalent VBA program, and one that should be used whenever complex numerical computations are required in your application. Microsoft Excel (and other spreadsheet programs) are unique among application development platforms in providing both a procedural (VBA) and a declarative (the worksheet) programming language. The most efficient Excel application is one that makes appropriate use of both these languages. It is assumed the reader of this book has some knowledge of Excel and worksheet functions, so Chapter 14 – Data Manipulation Techniques focuses on using advanced worksheet functions (including best-practice suggestions for handling circular references) and Excel's other data analysis features. The Excel Object ModelWhile the other four components of the Excel Platform are invaluable in the development of applications, it is probably the richness of the Excel Object Model that provides the most compelling reason to base our application development on Excel. Almost everything that can be done through the user interface can also be done programmatically by using the objects in the Excel Object Model – accessing the list of number formats and applying a digital signature to a workbook are perhaps the most notable exceptions. The vast array of functionality exposed by these objects makes highly-complex applications fairly simple to develop – it becomes more an issue of when and how to efficiently plug the functionality together than to develop the functionality from scratch. This book does not attempt to explore and document all the back-waters of the object model, but instead makes continual use of the objects in our application development. StructureThrough the course of this book, we will be both covering the concepts and details of each topic and applying those concepts to a time-sheet reporting and analysis application we'll be building. The chapters are therefore arranged approximately in the order in which we would design and develop an Excel application:
ExamplesThroughout the book, we will be illustrating the concepts and techniques we introduce by building a timesheet data-entry, consolidation, analysis and reporting application. This will be comprised of a data-entry template to be completed by each employee, with the data sent to a central location for consolidation, analysis and reporting. The end of each chapter will see a fully-working example of both parts of the application included on the CD, which will grow steadily more complex as the book progresses and thereby be applicable to different types of company. In Chapter 4 – Worksheet Design, we will start with a very simple data-entry workbook and the assumption that each employee would email the completed file to a manager who would analyze the results manually – a typical situation for a company with maybe 10-20 employees. By the end of the book, the data-entry workbook will use XML to upload the data to a web site, where it will be stored in a central database. The reporting application will extract the data from the database, perform various analyses and present the results as reports in Excel worksheets and charts. Along the way, we will be re-writing some of the parts of the application in a number of different ways, to show how easy it can be to include other languages and delivery mechanisms in our Excel-based applications. Each chapter may also include specific examples to illustrate key points that it would be too artificial to include in our main application. Supported VersionsWhen developing an Excel application for a client, their upgrade policy will usually determine which version of Excel we must use; very few clients will agree to upgrade their desktops just so we can develop using the latest version, unless there is a compelling business requirement that can only be satisfied by using features the latest version introduces. There is so little difference between Excel 2000 and Excel 2003 that it is hard to imagine such a business requirement. An extremely unscientific poll (based on postings to the Microsoft support newsgroups) seems to indicate the following approximate usage for each version:
There were a number of significant changes between Excel 97 and Excel 2000 for the application developer, including the switch from VBA5 to VBA6 and the introduction of modeless userforms, interfaces, COM Add-ins and support for ADO. We have therefore decided to use Excel 2000 as our lowest supported version and development platform, with our applications tested in the later versions. Most of the concepts detailed in this book apply equally to Excel 97, but our example timesheet application will use features Excel 97 does not support. Whenever we discuss a feature that is only supported in the later versions (such as XML import/export and VB.Net integration in Excel 2003), we will state which version(s) can be used. TypefacesThe following text styles are used in this book: Menu items and dialog text will be shown as Tools > Options > Calculation > Manual, where the '>' indicates navigation to a sub-menu or dialog tab. Sub SomeCode()
'Code listings are shown like this
'With new or changed lines highlighted like this
End Sub
Code within a paragraph will be shown like Application.Calculation = xlManual. Paths on the CD will be shown as \Concepts\Ch11 - Interfaces URLs will be shown as http://www.oaltd.co.uk Important points or emphasised words will be shown like this. On the CDMost of the code listings shown in the book are also included in example workbooks on the accompanying CD. For clarity, the code shown in the book uses shorter line lengths, a reduced indent setting, fewer in-code comments and less error handling than the corresponding code in the workbooks. The CD has three main directories, containing the following files: \Tools contains a number of tools and utilities developed by the authors which we have found to be invaluable during our application development. The MustHaveTools.htm file contains details about each of these tools and links to other third-party utilities.
Help and SupportQuestions about the book itself (such as missing CDs, typos, errata etc) should be directed to Addison Wesley, at http://www.awprofessional.com/contactus. Any errata and corrections will be posted to the Addison Wesley web site, at http://www.awprofessional.com/title/0321262506. By far the best place to go for help with any of your Excel development questions – whether related to this book or not – are the Microsoft support newsgroups archives maintained by Google at http://groups.google.com. A quick search of the archives is almost certain to find a question similar to yours, already answered by one of the many professional developers who volunteer their time helping out in the newsgroups, including all three of this book's authors. On the rare occasions that the archives fail to answer your question, you're welcome to ask it directly in the newsgroups by connecting a news reader (such as Outlook Express) to msnews.microsoft.com and selecting an appropriate newsgroup, such as
There are a number of web sites that provide a great deal of information and free downloadable examples and utilities, targeted towards the Excel developer, including:
FeedbackWe have tried very hard to present the information in this book in a clear and concise manner, explaining both the concepts and details needed to get things working and providing working examples of everything we cover. We have tried to provide sufficient information to enable you to apply these techniques in your own applications, but without getting bogged down in line-by-line explanations of entire code listings. We'd like to think we've been successful in our attempt, but encourage you to let us know what you think. Constructive criticism is always welcomed, as are suggestions for topics you think we may have overlooked. Please send an email to one (or all) of the following: Stephen Bullen:
stephen@oaltd.co.uk |
Reproduced with the permission of Addison-Wesley. All material copyright Pearson Education, 2005. No part of this material may be reproduced or transmitted in any form without the prior consent of the publisher.