Robert Watson on FreeBSD and TrustedBSD

Last Friday we solicited questions for Robert Watson, hard-core FreeBSD and TrustedBSD developer. His answers (below) are breathtakingly deep and instructive. Whether you're "just curious" about BSD or a FreeBSD user who wants to see what's going on with the inner circle of developers, you'll want to spend the time it takes to read everything here, and possibly even send Mr. Watson a brief "thank you" email.

OS X based on FreeBSD
by Kevinv

OS X's Darwin is based on FreeBSD. How good a member of the Open Source movement has Apple been? Have they contributed anything back to the FreeBSD project (code/money/t-shirts/etc...)?

Robert:

The easy answer is that Apple is involved in the open source community, and appears to be strongly committed to releasing their own software as open source, and contributing changes back to other projects whose software they use. Clearly, they're fairly embroiled in their upcoming release process at this point, but I'd expect more news on this front in the future.

They've had a strong presence at various technical conferences, including the BSD Conference in Monterey last year, and they're helping to sponsor and are participating in the Open Packages project. I've visited Apple on two occasions to discuss both FreeBSD and TrustedBSD work with them, and had the opportunity to meet with many of the people in their Core OS Group. While I don't know everything they've been up to, I can speak to their shipping me two iMacs so I could explore the operating system and look at porting some of the TrustedBSD work to it, and must say that

I'm very impressed.

One thing I think the FreeBSD project should do is select a liaison to work with Apple to help them understand our development model better, and help integrate back changes made to Darwin. Especially in light of all the changes coming in FreeBSD 5.0, it's important that we work together to prevent substantial divergence between our source trees (where possible) allowing us to continue to exchange features in the future. I have to give Apple a big thumbs up, and hope they keep up the good work!

what do you do for *money*??
by gskouby

While perusing the mailing lists for -hackers, -stable, -current, etc. etc., I often wonder what people like yourself, Mike Smith, Greg Lehey, and the other core members do to pay the bills.

Unless something has changed recently with the BSDi takeover, I can't imagine that the FreeBSD project keeps the food on the table. So how about a little insight into your and the other core members "real" jobs. (As if there is such a thing as a "real" job). But anyways, thanks for all the hard work for little pay!

Robert:

Ahh, the perils of capitalism. Needless to say, all core members enjoy employment in some form or another. Some work with companies that pay them to work on FreeBSD (including BSDi), others do independent consulting on (among other things) FreeBSD; others still work on utterly unrelated areas. Since the question was raised, I'll talk a little about what I do, and how it does relate to FreeBSD.

I work for NAI Labs, a research organization that is now associated with PGP, Inc -- about 100 full time researchers doing advanced research and development for the likes of DARPA, other government agencies, industry, and internal research and development. If you don't recognize the name, we used to be the Advanced Research and Engineering (ARE) division of Trusted Information Systems (TIS). At NAI Labs, I'm a Research Scientist in the Network Security research group, and have worked on a variety of projects including securing DNS (DNSsec), DHCP security, active network security, and denial of service research. While most of my work (right now) is relatively unrelated to FreeBSD, we hope to change this in the relatively near future, identifying funding for work on FreeBSD and TrustedBSD, as well as porting work to OpenBSD, and work on Darwin. Other examples of operating system security work on open source here include Software Wrappers and Low Watermark Mandatory Access Control. One of the great things about working at NAI Labs is the opportunity to participate in cutting edge security research, and the opportunity to set your own direction. All in all it's a really nice place to work, and I recommend it highly--in fact, we're actively hiring at this point, so if you're interested, feel free to fire off a resume to [e-mail].

Of course, companies can greatly benefit from employing a FreeBSD developer, as they have the opportunity to influence development of the operating system (subject to the common sense of the developer and consensus of the project as a whole, needless to say). Many FreeBSD developers, looking at the committer community as a whole, are employed to do what they would like to be doing anyway: working on a section of the system that interests them.

TrustedBSD and NSA secure linux
by Xuther

How does TrustedBSD compare with NSA secured linux (http://www.nsa.gov/selinux) in terms of new and or improved security features? And are there any plans to eventually integrate the rest of the TrustedBSD features back into the shared BSD source tree (the extended attributes already have been committed)? How would using TrustedBSD instead of FreeBSD impact clustering applications?

And just for my information, where did all the packages for clustering BSD go? All I can seem to find anymore is the linux stuff. And personally I don't like redhat and their rpm distribution method, all anyone wants to distribute anymore is rpms which is not near enough to standard and compatable accross the board as tar-gzip for my purposes. (One primary difference being that I can open a tar-gzip on a windows box at work during break to browse through source, and to my knowledge no one has bothered to create a "winrpm")

Robert:

These compound questions are the killers :-).

I am both aware of and familiar with the NSA Secure Linux work -- a fair amount of the work is being done at NAI Labs under contract from NSA. Stephen Smalley, one of the lead developers on the project, actually works just upstairs from me in the Glenwood, Maryland office of NAI Labs. As such, I've had a number of opportunities to talk with him about the work. One of the advantages of working at NAI Labs is the ability to get wide exposure for a variety of security-related research on many platforms, and relating to many topics.

TrustedBSD and SELinux are similar in many ways, and also differ in many ways. The similarities lie in overlapping functionality and architectural goals; the differences only begin with the choice of operating systems. TrustedBSD introduces a number of features into the FreeBSD operating system including Mandatory Access Control (MAC).

In a broad generalization, MAC allows administrators to define security policies about how users interact with one another. These policies are mandatory in the sense that users are not permitted to change the policies, although some flexibility may be permitted by the policy.

MAC is distinguished from Discretionary Access Control (DAC) in this manner; most Linux or FreeBSD users will be familiar with DAC in the form of file permissions. In DAC models, the owners of objects (and possibly other parties) are permitted to modify protections to reflect their needs. A common mandatory policy is Multi-Level Security (MLS), or the "military security model".

In this model, users are assigned "clearances", and objects are assigned "classifications". MLS prevents users from reading files they are not allowed to read, but it also prevents users from sharing files they are not allowed to share (this is the mandatory component). MLS is just one mandatory policy, there are many others that have been defined and explored in various environments. TrustedBSD implements three mandatory policies in its current prototype form: MLS, a Biba integrity policy that is similar to MLS but protects integrity instead of confidentiality, and a light-weight partitioning scheme that is an extension of the popular jail() mechanism introduced in FreeBSD 4.0-RELEASE.

SELinux provides an implementation of a flexible mandatory access control architecture called Flask for Linux. The architecture is a generalization of Type Enforcement (TE) and can support a wide variety of mandatory security policies. In the Flask architecture, the security policy is encapsulated in a pluggable "security server" component that can be replaced. The example security server provided with SELinux includes support for TE, MLS and a simple form of Role-Based Access Control (RBAC). An important focus of this architecture is separating policy representation and processing from policy enforcement.

SELinux differs from TrustedBSD in that it is a more mature system, having been worked on for several years, that it addresses only mandatory access controls, and that it uses the Flask architecture rather than explicit hard-coded policies. It is certainly the case that we plan to leverage the SELinux implementation now that the source code is available; the abstractions of the Flask architecture are similar to ones that were being considered for TrustedBSD. Having the opportunity to look at the SELinux source will allow us to benefit from their implementation experiences.

As you observe, some TrustedBSD features have already been integrated into the base tree, including extended attributes on files, as well as infrastructure support for capabilities, ACLs, and some of the improved abstractions I spoke about above. The plan is to integrate most of the TrustedBSD features into the base operating system distribution over time; some features are more intrusive, as well as more computationally expensive, than others, meaning that some features may be distributed as modules rather than enabled by default. However, it is a definite goal to make all of the work easily available for FreeBSD installations, and under a two clause BSD-style license. Many of these features will appear in FreeBSD 5.0-RELEASE, although they will presumably mature over time.

The remainder of your questions address clustering; I have to begin by pointing out that I don't have much experience with clustered environments. I can probably safely comment that the TrustedBSD features won't present any substantial additional impediment towards implementing clustering, either in terms of functionality or performance. Most of the of the TrustedBSD features either supplement base UNIX features without substantially changing them in ways that impact applications, or are disabled unless specifically configured.

My understanding is that many of the normal computational clustering tools, such as PVM, are available via the FreeBSD ports/packages collection, and that FreeBSD is used in clustering, but as a non-expert can't speak much to this issue. As clustering means something different to every observer, this may not have answered your question, and I'd welcome follow-up e-mail to discuss this further.

Openpackages?
by Enahs

What's your opinion on the Open Packages project? Even though I'm not currently a *BSD user, it sounds great on the surface--there's even been interest expressed in patches for Linux!--but I've got to wonder what sort of complexities need to be worked out to maintain a set of packages for FreeBSD, NetBSD, OpenBSD, Darwin...

Robert:

I think OpenPackages is a great idea: the ports/packages on FreeBSD and other BSD platforms have been an incredible boon for the users of these systems. One of the disadvantages of BSD is that it hasn't presented a single porting target, and that there has been redundant porting work going on. OpenPackages offers a way to reduce redundant work, and improve application portability. I was excited to see Apple on the list of sponsors for the project, it shows continued commitment by Apple to open source.

A few important questions:
by Bob Abooey

1) Do you ever plan on moving away from the slow and resource intensive method of VMS style paging for memory address resolution

Robert:

I'm not sure how to interpret this question; FreeBSD provides the standard UNIX-like API for memory management (brk(), sbrk(), memory mapping, protection modification, SysV shared memory). This is supported by a Mach-based virtual memory system that has undergone substantial feature evolution and performance optimization. All performance benchmarks I've seen suggest that the FreeBSD virtual memory system is both robust and high-performance under both light and heavy loads. This is one of the features of FreeBSD that has made it so popular for web farms and file servers.

My understanding is that the new VM system under development for Linux takes into account the FreeBSD VM design, and shares many of its design choices and, as a result, performance and stability properties. However, I have not followed that work closely enough to comment in great detail on the topic.

2) Are there plans to rewrite the TCP/IP stack to be multi threaded

One of the major development projects currently underway is "SMPng", or the Next Generation SMP project for FreeBSD. The SMPng project goals include:

• A fully preemptive and reentrant kernel
• Fine-grained data based locking
• An evolutionary development process
• Rapid development cycle leveraging technology donated by BSDi from their next generation SMP support under development for BSD/OS, including debugging tools and operation models
• Thread-based interrupts allowing blocking at will

This should substantially improve performance on SMP machines, as well as modernize the structure of the kernel. It will include work to push down locks (eliminating the giant kernel lock present in other versions of FreeBSD), including in the network subsystem, allowing components of the network stack to execute in parallel on different processors.

3) Will BSD ever migrate away from UFS to a more modern file system?

It depends what you mean by a "modern file system". Right now, FreeBSD actually uses FFS, the Berkeley Fast File System, with the addition of "soft updates" for performance and consistency, and under 5.0-CURRENT (the development branch), the ability to atomically snapshot file systems, as well as the ability to store extended attributes on files, in turn supporting other features such as Access Control Lists (ACLs). Fsck-less booting is currently a work in progress, and will be in 5.0-RELEASE also. In fact, several sites including Yahoo! have already deployed fsckless booting internally. paper presented at the USENIX 2000 Technical Conference discusses the performance and consistency differences between journalled and soft updates consistency mechanisms. The paper in question also discusses two different journalling implementations based on FFS and developed on FreeBSD, which will be made available at some point to the FreeBSD project for possible integration. I think it's safe to say that, by most definitions, FFS on modern BSD operating systems is very much a modern file system.

4) With serious POSIX compatablity issues are there plans to use code from POSIX compliant OS's to become more commercially attractive to major corporations

One of the FreeBSD Project goals is to comply with appropriate API and user interface standards. Generally speaking, a failure to comply with a relevant standard is considered to be a bug, and should be reported using the standard bug reporting tools (we use GNATS to track bugs). If you are aware of non-compliant features or interfaces, please let us know and we will endeavor to fix them.

Why would you... ?
by SonOfSam

FreeBSD development is obviously a big part of your life. I have noticed that peoples reasons for using a free OS are often not simply because its better, but because of some view or stance on freedom that they have.

I am a Windows guy, only because my job says so.

What I want to know is, how would you go about convincing me, a Win2k user, to consider using a *BSD. I am interested in learning a new OS... always. But, what makes it stand out from Linux/Win2k/MacOS?

Robert:

There are a number of aspects to your question, and there are a lot of ways I could explore it. It seems that the first part of the question relates to why I as a user and developer make use of FreeBSD (and open source in general). Another aspect seems to be a question about political motivations.

Open source gives me as a developer the ability to do things that closed source could never allow--I get to determine what features are important and dedicate resources to making them available. As "extensible" as closed source operating systems may be, it is hard to understand how a system works without access to the source, and hard to modify it to do things the designers didn't anticipate. This argument is also a reason why NAI Labs does a great deal of its research and development on open source systems: it's simply impossible to get that level of responsiveness from a closed source system vendor.

As to political motivations? My motivation for pushing FreeBSD is the philosophy of the project, rather than general intuitions about personal freedom. The project (as with many open source projects) has a dedication to technical excellence and openness (of process, as well as source) that is outstanding. I wouldn't force anyone to open source their software as that's a personal (or often corporate) choice, but I recommend open source software widely.

One of the political aspects to open source is the selection of license: I don't see this as a big thing. The BSD license probably does better reflect both my beliefs and needs, but I use and modify software under a variety of licenses, and recognize that the license you release your software under has to reflect your own beliefs and needs. Any other understanding of license selection as a moral argument fails to recognize a contemporary understanding of relativism that is vital to cooperation :-).

As to why FreeBSD as opposed to any other operating system? Well, as I mentioned above, the FreeBSD project has a dedication to technical excellence and openness. What does this mean? It means that I have a high level of confidence in the software (both by reputation, experimentation, and source code inspection). The operating system is stable and performs extremely well, is extensible, and is being actively developed in a variety of ways. There's an attention to detail, as well as the big picture, that reflects a high level of dedication among developers.

Advantages over the traditional Windows platforms have long been stability and performance, as well as openness. I haven't run the performance numbers recently, but understand that Microsoft has made large investments in stability and performance for Windows 2000; it would be a mistake to underestimate their ability to improve in this area. As long as Microsoft remains closed source, however, they will be unable to match the openness that is vital to the work that I do.

The primary difference that I see as important for me when comparing FreeBSD and Linux lies in the development model: there is a central forum and structure for the FreeBSD developer community that provides a forum for communication, group decisions and consensus building. My feeling is that this leads to better design decisions, and a focus that reflects a whole-system view. An important question for the FreeBSD Project as we move forward is whether or not this model can scale easily as we expand. The number of "committers" on the FreeBSD team has dramatically expanded over the last couple of years; many of these developers are working on the ports/packages and documentation, but many of them also work on the base system. Moving towards an elected core team, as well as ongoing debates on the development model and source code management reflect the increasing size and more diverse needs of the developers. The SMPng project's managed development model is another sign of this growth, and an example of a successful attempt to address the need for more structured development practices in the face of a larger audience and more people working on the same code.

Mac OS X appears to have a bright future: Apple has managed to tread in NeXT's footsteps when it comes to combining a mix of strong technical components from the open source and research communities, as well as excellent internally developed work. OS X represents a number of dramatic changes for the Mac user community; Apple has in the past shown a great deal of responsiveness to that community, and OS X looks like it will be an interesting fusion of advanced operating system concepts and a highly usable interface. Part of what will be important in the widespread adoption of Mac OS X is consistency with prior versions, allowing users to migrate in a relatively seamless manner. In Steve Jobs' recent demo and presentation, inclusion of the traditional Apple Menu appears to demonstrate sensitivity to this issue, and responsiveness to the comment submission process. I see a place for a Mac OS X box on my desk in the near future.

Because it leverages FreeBSD work, and because FreeBSD leverages Apple's work, I don't see them as mutually incompatible. It is my firm hope that Apple and the FreeBSD Project find ways to work together more in the future, because I think everyone will benefit from this.

These are just my opinions, and I would expect others to disagree with me. I should point out that in the past, I've recommended the use of a variety of operating systems to both individuals and companies; this includes Windows, Linux, Mac OS, Solaris, and FreeBSD. I'd be a fool to assert that any operating system is appropriate for all uses and environments.

decent literature
by boog3r

instead of asking you a few questions directly, i would like to solve them on my own with the best set of tools. what publications or literature would you recommend for:

the *bsd newbie or learner
the *bsd uber-know-it-all-i-dont-need-any-docs

i am trying to cut the signal/noise ratio out of understanding bsd. specifically, what security documentation have you found useful day-in/out?

Robert:

I'll speak to the FreeBSD section of BSD, since that's what I'm most familiar with. There are several books available describing FreeBSD.

The most commonly used is The Complete FreeBSD by Greg Lehey, which can also come bundled with a CDROM set, making it easy for new or experienced users to go to a single source. The book is currently in its third edition, and apparently there is a fourth edition currently under preparation. I saw this book in the local Barnes and Noble's recently, so it should be fairly easy to locate. A recent addition to the collection of books on FreeBSD is Ted Mittelstaedt's The FreeBSD Corporate Networker's Guide, which also includes a CD-ROM.

The online documentation for FreeBSD is also strong, both in the form of the FreeBSD Handbook, which includes both reference and tutorial materials accessible via a web browser, and the normal UNIX-like man pages. The FreeBSD Handbook is also available in printed form. Both are actively maintained and regularly extended to cover new features.

FreeBSD and BSD books are generally available from BSDi via their FreeBSD Mall Books page, and from the Daemon News Books and Posters page, not to mention your normal online book vendors (Amazon, Barnes and Noble, et al), and quite possibly your local bookstore.

In addition, the BSD Daemon News magazine is now available in print as well as electronic form, and includes articles appropriate for both users and developers.

I suspect the "uber-know-it-all-i-dont-need-any-docs" guy is unlikely to listen to any recommendations from me, but would probably find the man pages most useful as they're more reference than tutorial :-).

To be honest, I don't use security documentation other than the man pages, as I'm familiar with most of the base system features, as they're an area where I've done a lot of work. Out of the box, FreeBSD is a fairly safe beast, as long as you've reviewed recent security advisories for the release you're using. The usual advice applies: don't install or run things you don't need to, and emain up-to-date on security advisories. There's a FreeBSD security how-to on the web site.

For the die-hard "uber-know-it-all", there's always the source, which in the end is authoritative as to how the implementation works, regardless of documentation :-).

----------------

I noticed in the comments, although it didn't make it into the questions in the interview, that there was a lot of curiosity about the relationship between the OpenBSD Project and the TrustedBSD Project, also regarding TrustedBSD and FreeBSD. As it's important to understanding the work I do, and the goals of the project, I figured I should throw in a bonus answer:

TrustedBSD provides a set of extensions to FreeBSD to add support for {ACLs, Capabilities, Mandatory Access Control, Auditing} as well as supporting features to implement them. As I described above, these features are being integrated into the base operating system distribution, with the intent that they be "part of FreeBSD". This is facilitated by having some of the TrustedBSD developers also be FreeBSD Project developers.

The OpenBSD and TrustedBSD projects have largely different thrusts: while the OpenBSD project seeks to provide a correct and bug-free POSIX implementation (where correctness includes a focus on failing to suffer from security holes). It also includes cryptography-related features as a primary development goal, hence early development and integration of IPsec in the base system (and a continuing high level of maturity of their implementation), as well as their work on OpenSSH. The TrustedBSD project seeks to introduce a variety of features, some described in the defunct POSIX.1e draft.

While TrustedBSD targets FreeBSD as the starting operating system, it should be observed that all of the BSD systems stem from the same source base, and remain very similar. This means that porting a feature from FreeBSD to OpenBSD should prove relatively straight-forward. The same goes for Darwin, the kernel from Mac OS X. I list both of these explicitly because we in fact have plans to start porting features to both of these platforms, as resources permit. The starting point in both cases will be to make Extended Attributes available in their file systems; these are used to store the supporting data for ACLs, capabilities on files, and MAC labels. I'd welcome interest in porting these features to other BSD platforms, including NetBSD and BSD/OS as well.

---------------