Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
BSD Operating Systems

BSD Journaled File System Ready For Testing 113

Posted by timothy from the dear-diary dept.
Dan writes "The Journaled File System for FreeBSD (JFS4BSD) Project has the goal of porting the JFS Technology from IBM/Linux to FreeBSD. It uses a log-based, byte-level file system that was developed for transaction-oriented, high performance systems. Scalable and robust, its advantage over non-journaled file systems is its quick restart capability: JFS can restore a file system to a consistent state in a matter of seconds or minutes. The jfsutils is under a compilable state on FreeBSD."
This discussion has been archived. No new comments can be posted.

BSD Journaled File System Ready For Testing More

BSD Journaled File System Ready For Testing

Comments Filter:

  • A mixed blessing. (Score:5, Insightful)

    by cbiffle ( 211614 ) on Sunday February 16, 2003 @05:11PM (#5315678)
    This strikes me as both a good and a bad thing.
    Good, because journalled filesystems are a Good Thing. I use FreeBSD exclusively on both desktop and workstation, and while SoftUpdates is very good, it and journalling have different aims. (SoftUpdates aims to keep all file metadata consistent at all times; journalling aims to keep all file -data- consistent at all times.) Choice is always good, and I could see myself using SoftUpdates on /home and JFS on the maildirs, for example.
    However, this is a bit of a Bad Thing in that one of FreeBSD's simplicities has always been the One Filesystem, UFS. Granted, UFS has evolved some (UFS-Softupdates and now UFS2) but there was never a question as to which filesystem to choose: UFS has enough tunables, most of them automated, that you can optimize it for small-file, large-file, high-latency, low-latency, etc. I've found it to be capable of keeping up with the various Linux filesystems in their own areas. But if this is merged into -current, there will be a choice to make when preparing a slice. This is one of those things that's hard to change after the fact. :-)

    As for me, I'll stick with UFS2 until I see how this shapes up, but tally-ho!
    • so you basically what you are saying is that having a choice is a "bad thing". and in other completely unrelated news,the average iq of slashdot posters takes yet another nose dive.

    • A gentle correction... (Score:5, Informative)

      by plsuh ( 129598 ) <plsuh@nOSPAM.goodeast.com> on Sunday February 16, 2003 @06:03PM (#5315861) Homepage
      JFS, other journalled file systems, and Softupdates have the same goal -- keeping file metadata structures consistent on the disk. JFS does not attempt to maintain file data integrity in the event of a crash -- that is the job of a DBMS. Go and read the web page on JFS [ibm.com] from IBM that is linked to in the original posting.

      Granted, journalled FS's and softupdates go about things in different ways. Softupdates trades off potentially increased disk space usage and higher disk and CPU activity after a crash (performing the background, reduced set of checks in fsck) against a smaller relative performance penalty vs. a non-journalled FS in normal usage as compared to a journalled file system.

      My own $0.02 is that this is a nice scratch-the-itch, check-the-box-for-PHB's addition, but for most normal usage softupdates is a better choice. See the papers by Ganger and Patt [cmu.edu] and McKusick [mckusick.com] for more details. (Links copied from the OpenBSD FAQ pages.)

      --Paul
      • I did not quite understand your second paragraph. Could you please explain the advantages of softupdates versus journalled again. thanks.

        • Re:please explain softupdates vs. journalled fs (Score:5, Informative)

          by plsuh ( 129598 ) <plsuh@nOSPAM.goodeast.com> on Monday February 17, 2003 @08:10PM (#5322681) Homepage
          My understanding is that a journalled FS records every file block allocation and deallocation on the volume in a separate journal. This way, if an operation does not complete (say, due to a power failure), the disk can be restored to a known consistent state very easily. A consistent state is one where no blocks are allocated that should not be, and no blocks that are in use are listed as unallocated.

          Softupdates does a partial re-ordering of operations so that all allocations happen before deallocations. Thus, in the event of a crash there can never be any blocks that are in use but marked as unallocated. The only possible inconsistency is that there may be blocks that are unused but marked as allocated. In the event of a crash, no fsck is performed as a part of the boot sequence, but a special background version of fsck is run to find any blocks that unused but marked as allocated and mark them as unallocated, which is a safe operation.

          Comparing the two methods of operation, a journalling FS pays a speed penalty under normal operations, as it needs to do two writes to the volume for every write access from the application, one to write the journalling data and one to write the actual write. However, on a restart after a crash, the journal is just used to roll back the volume's allocations to a known good state and the OS continues on its merry way.

          FFS with softupdates is faster under normal operations, as there is only one write to the volume for every write access from the application. However, in case of a crash there is the potential for a large quantity of disk blocks that will need to be scavenged, thus making the volume look like it has less free space than it actually has, plus the background fsck needs to run and fix up the allocations.

          Read the Ganger and Patt and McKusick papers linked to in my original post for more details.

          --Paul
          • Ah thank you. That makes sense.
          • great explaination. thank you.

            there is one thing i don't understand. when you do a freebsd installation, why is the default root (/) filesystem not marked automatically for using softupdates?

            is there a cause for not using it under special circumstances?
            • No idea. I'm not part of the FreeBSD team, and in fact I'm more of an OpenBSD user/sysadmin. Try asking on one of the FreeBSD lists.

              --Paul
            • This decision was made due to space concerns. For users with small filesystems, it was a common problem, during an installkernel, to run out of space on / because the previously unlinked blocks from things such as /modules/ or /boot/kernel/ were not deallocated yet (deallocation can take time using softupdates, but happens eventually). The solution to this problem was to have the root filesystem default to not having softupdates. This was just due to the fact that you do not want your root filesystem filling up, and this really made it a pain for many users to upgrade. Perhaps a better decision would be to have sysinstall toggle it based on how large the filesystem is, but it's really a tough call. But anyway, there's some fairly recent history for you :)

              Cheers,
              -JD-
      • From what I could tell from reading those papers, softupdates can't recover if the corruption occurs while writing a sector to the drive. With that kind of error a journal is necessary to be able to recover. The other solution would be to always rewrite the superblock for every update and maintain several copies of the superblock (effectively journalling only the superblock.) This is essentially what TUX2 does.

        I could be mistaken, but my understanding is that softupdates may allow you to update an on-disk inode or directory structure that has other things pointing to it, which could corrupt a filesystem if the system crashed during the update.

        What's your understanding of this? Am I mistaken?

    • Re:A mixed blessing. (Score:5, Informative)

      by ctr2sprt ( 574731 ) on Sunday February 16, 2003 @06:13PM (#5315889)
      (SoftUpdates aims to keep all file metadata consistent at all times; journalling aims to keep all file -data- consistent at all times.)
      That's just not true. Soft updates works by ordering writes so that the metadata is always in a consistent state. This has the effect of also keeping the data in a consistent state, since data sits in the to-be-ordered queue right along with the metadata. Journaling filesystems typically track only the metadata (though some, like e3fs, can also do ordered data writes), which means that data can be lost as the metadata is returned to a consistent state.

      Don't believe me? Read the JFS overview from IBM [ibm.com], which says:

      JFS only logs operations on meta-data, so replaying the log only restores the consistency of structural relationships and resource allocation states within the file system. It does not log file data or recover this data to consistent state. Consequently, some file data may be lost or stale after recovery, and customers with a critical need for data consistency should use synchronous I/O.
      The only benefit JFS has over UFS+SU in this context is faster on-crash fsck times. And it only takes about a minute to check a 60GB UFS filesystem after a crash, which is about 3 orders of magnitude better than e2fsck (I don't know how speedy jfs.fsck is in non-replay mode).

      The chief benefit of JFS for FreeBSD is going to be portability. Which is a pretty big one, although I don't see JFS supplanting UFS before FreeBSD 6.0, if then.

      • Re:A mixed blessing. (Score:2, Informative)

        by Nevyn ( 5505 )
        That's just not true. Soft updates works by ordering writes so that the metadata is always in a consistent state. This has the effect of also keeping the data in a consistent state, since data sits in the to-be-ordered queue right along with the metadata.

        wrong, if you write to the middle of a file then both UFS+SU and a metadata-only journal won't keep the data consistent in a crash (Ie. you can have some old data and some new data).

        If you have a real journal (Ie. data=journal in ext3) then you get consistent data writes too, which is very different. There are other ways to achieve the same thing (WAFL, the unfinished Tux filesystem for Linux) you appear to be correct that JFS only supports a metadata journal.

    • The nice thing about UFS in FreeBSD 5.0 is that you have background fsck. So the machine boots really fast and if you have unclean filesystems they will be checked in the background. You can work with your computer in the mean time, you will just notice a slight performance drop.

  • ReiserFS (Score:2, Redundant)

    by DiSKiLLeR ( 17651 )
    Whats wrong with ReiserFS (my favourite) or EXT3? or SGI's XFS even? Strangely, i've never even heard of IBM's JFS before, as ReiserFS/EXT3/XFS make all the news here.

    D.

    • Re:ReiserFS (Score:2, Interesting)

      by rplacd ( 123904 )
      jfs is one of the oldest journalling filesystems available. it's stable, has been stress-tested, is backed by ibm, etc.

      reiserfs and ext3fs are nowhere near as capable. xfs is pretty good, too -- in fact, it's what i use on my debian box.

    • Re:ReiserFS (Score:3, Insightful)

      by Arandir ( 19206 )
      ...as ReiserFS/EXT3/XFS make all the news here.\

      And here you have discovered the biggest diffrerence between Linux and *BSD. BSD developers don't give a flying fsck about making the news, they just want good solid stable code. I mean, who out there with more than two bytes of valuable data gives a rat's arse about which filesystem makes the news the most?

    • Re:ReiserFS (Score:3, Funny)

      by Sygnus ( 83325 )
      Whats wrong with ReiserFS (my favourite) or EXT3?

      The GPL.

      • Explain why a GPLed filesystem is bad.
        • Because it's infrastructure. Such stuff should be under the LGPL, BSD, MIT or similar unrestricted or weak copyleft licenses. Even the Linux kernel has an exception that moves its license to the state of weak copyleft.

  • is it gpl'd? (Score:2, Interesting)

    by rplacd ( 123904 )
    if so, it'll never end up as part of the base install.

    • Re:is it gpl'd? (Score:2, Informative)

      by Ded Bob ( 67043 )
      Yes [ibm.com].
    • Maybe not enabled by default, but that doesn't mean it won't make the base install. That didn't stop GPL_MATH_EMULATE :-)
      • GPL_MATH_EMULATE was NOT under the GPL!

        It is under a modified BSDL. How do I know? I'm the one that put it into FreeBSD way back when.

        Here is the clip from the top, it then follows up with a modified 4 clause BSDL (It's almost LGPL).

        This copyright notice covers the redistribution and use of the FPU emulator developed by W. Metzenthen. It covers only its use in the 386BSD, FreeBSD and NetBSD operating systems. Any other use is not permitted under this copyright.

        It was added to the system along with the sun libm so I could use ghostscript on a i386 with no math-co...:)

        BWP

  • UFS2, XFS, JFS, Vinum, FreeBSD, JunOS, everything! (Score:4, Insightful)

    by Zeio ( 325157 ) on Sunday February 16, 2003 @05:26PM (#5315738)
    One of the most impressive things about Linux is the XFS filesystem for it. It is thoroughly tested, production quality, and extremely robust. It is a please to deal with XFS. I personally have tested XFS for robustness and speed, and I have done things such as set up scripts to get, untar, copy erase, and start and stop various daemons and databases and then while this looping miasma was going on RIP THE HARD DRIVE from the system (SCA, SAF-TE, hop swappable). I have yet to see a lengthy FSCK or a corruption. Simply the best!

    UFS2 is interesting, and provides a lot of improvements to UFS1, the soft updates are fairly effective at keeping things consistent and there is a background fsck in FreeBSD that works quite well.

    A filesystem which is just as robust as XFS in terms of durability is JFS, but sadly I have found this filesystem to be a bit short on the performance side. I believe the main function of JFS is to provide support for those moving from older IBM systems to newer things that possibly include the us of Linux over AIX or OS/2.

    I personally will not consider Reiser or EXT3 and could go into detail as to why. I have strong opinions as to what types of filesystems belong in production, and these will not qualify.

    So, while JFS for FreeBSD is a good thing, I would like to see at some point an attempt to move XFS to FreeBSD, and if I had that capability, I would have already started.

    FreeBSD is coherent, well documented and with things such as Vinum, and JFS-like filesystems and hopefully someday XFS, it really does quote a bit and is released under a flexible, corporate-friendly license (which I believe helps to foster and promulgate further development, not stifle or prevent it).

    On of the series of equipment that I am most impressed with is the Juniper routers. They are the best routers available in my estimation (65xx switches are my most favored switches). I feel very at home with JunOS because it is largely FreeBSD:

    JuneOS version 5.6R1.3 built by builder on 2003-01-02 20:19:20 UTC
    Copyright (c) 1996-2001, Juniper Networks, Inc.
    All rights reserved.
    Copyright (c) 1992-2001 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
    JUNOS 5.6R1.3 #0: 2003-01-02 20:38:33 UTC
    builder@zu.juniper.net.:/build/zu-b/5.6R1.3/o bj-i3 86/sys/compile/JUNIPER

    • Re:UFS2, XFS, JFS, Vinum, FreeBSD, JunOS, everythi (Score:5, Funny)

      by battjt ( 9342 ) on Monday February 17, 2003 @08:47AM (#5318747) Homepage
      I personally will not consider Reiser or EXT3 and could go into detail as to why. I have strong opinions as to what types of filesystems belong in production, and these will not qualify.

      How can a stupid statement like this get moderated to the top? Unsubstantiated claims like this should only be allowed to criticize Microsoft. Comments critical of Linux must be backed up with hard data.

      I personally will not consider this comment and could go into detail as to why. I have strong opinions as to what types of comment belong in /., and this will not qualify.
    • with regard to XFS
      I have yet to see a lengthy FSCK or a corruption.

      Lord knows I have. I had a power outage at my house and got an error on boot for my /home partition. When I searched google with the non-unique part of the error message, the first link on google was a message on the xfs mailing list by one of the developers saying he corrected an endian bug that was probably causing people to have recovery failures. Ah, here [theaimsgroup.com] it is. I couldn't find anything on the web/newsgroups about how to fix it. I think I eventually ended up using xfs_repair. It complained about the filesystem not having been unmounted cleanly and telling me to mount and then unmount the filesystem. Of course I couldn't mount the filesystem anymore. I can't remember what exactly it told me to do in the event that the filesystem could not be mounted, but it involved some scary warning about losing the filesystem. I think it was to use xfs_repair -L (zero out the journal log). I considered creating another partion (I don't need as much space as today's drives ship with so I leave space to create new partitions/grow old ones) and using xfsdump to back up the filesystem at that point, but there was nothing terribly important on there so I just went for it. It repaired the file system just fine. I've had it put stuff in lost+found for some of the other file systems on other power outages (yes, we have lots of power outages and I need to buy a UPS though we don't seem to have had one for 43 days at this point), but that's not really a big deal. It does mostly restore things to how they are suppose to be.
      The problem with xfs is that they don't release "stable" patches for 2.4 that often. Oh, they release snapshots for every kernel, but those are at the start of the branch to that kernel and not at the end of the branch. Other than that you can pull their linux kernel with xfs out of their cvs, but they do not have any sort of stable branch in their cvs that I can find. There were 9 months between the release of xfs linux release 1.1 and 1.2 during which time many important bug fixes went into cvs.
      Don't get me wrong. I still use xfs as my filesystem. Mostly because they have actual real acl's. I just wish they would release some sort of "stable", "blessed" version of their patches every now and then. Can't wait for 2.6 when it's just included in the stable kernel.

  • a useful thread to read (Score:2, Interesting)

    by rplacd ( 123904 )

    see this [slashdot.org] thread on journaling filesystems & freebsd.

    i'd like to see xfs on freebsd too, but the license...

  • *BSD filesystem group (Score:4, Insightful)

    by hubertf ( 124995 ) on Sunday February 16, 2003 @07:42PM (#5316219) Homepage Journal
    I think it would be nice to have a group working on filesystems for all BSD systems (Free, Net, Open, ...), just like what KAME does for IPv6 on BSD.
    That way, every BSD project would benefit from the efforts.

    - Hubert
    • I know NetBSD has LFS (log-structured file system), which I assume is journaled. Does anyone know anything about this? I know it's been getting some attention from the developers lately after a period of being stale.

      Cheers,
      -JD-
      • To correct myself, it appears that log-structured filesystems are not the same as journaled filesystems..

      • LFS (Score:1)

        by britt ( 50456 )
        LFS (originally by Ousterhout, ported to 4.4BSD by Seltzer) writes everything in a log. Just write as much as possible in one long stripe to the disk (up to a meg in the initial designs.) This means blocks of data move each time they are updated, which means you always write new inodes with each data write which point to the old unchanged blocks, and the new changed blocks.

        Obviously once you hit the end of the disk, you have to wrap back around, and write some more, but where?

        LFS had a seperate backgroup process that went around and finds all the, now unneeded, old data blocks that have been rewritten somewhere else and cleans them up. This is called the cleaner. Once the cleaner has found more space you get to write there.

        The main positive point of the LFS design was very good write performance due to the lack of seeking needed to put little bits of data all over the disk.

        The main downside to LFS was the cleaner itself, which was costly to run.

        Margo Seltzer and Trevor Blackwell did some more work with the cleaner in this paper [tlb.org] that shows that with the right timing you can get the cleaner to run efficiently.

        I'm not aware of this work having been implemented (at least not as of 2000 which I stopped working on it on BSDI where Margo had been keeping LFS)

        A good description can be found in the original papers, and the Pink 4.4 BSD book [amazon.com] by McKusick, Bostic, Karels, and Quaterman.

        Britt

  • Quotas? (Score:3, Insightful)

    by nocomment ( 239368 ) on Monday February 17, 2003 @01:30AM (#5317597) Homepage Journal
    Will it do quotas for users?
    FreeBSD makes an awesome mail server for that reason alone (not that there isn't a ton more).

  • According to the comment on http://daily.daemonnews.org/view_story.php3?story_ id=3539 the JFS code itself isn't ready yet, but the jfstools are. So you can take a JFS disk from a Linux box and use the jfstools, but you can't mount the JFS right now.
  • I thought that one of the biggest reasons that the BSDs had not incorporated the publicly available journaling filesytems (SGI XFS and IBM JFS, and to a lesser extend ext3 and reiserfs) was that all the code was GPL, not BSD. I may be revealing my gross ignorance of kernel programming, but it was my impression that having GPLed filesystem code in a BSD-based system would not really be possible. How do the developers plan to get around this?

    • Re:GPL? (Score:1, Insightful)

      by Anonymous Coward
      GPL'd code can be included in any BSD project, but it must not be statically linked.

      So, this means that your boot (root) filesystem can never be JFS/XFS/ResierFS/ExtFS enabled, since the code to read your boot filesystem must be statically linked into the kernel.
      • As far as I know, the static linking restriction applies to the LGPL, not the GPL. The restriction in this case is that it can't be compiled in the kernel with the normal distribution, but they can include the source if they want, and you can compile your own kernel with any GPLed software linked in it.

        • Re:GPL? (Score:1, Insightful)

          by Anonymous Coward
          You've actually got it backwards. The difference between the GPL and LGPL is that the LGPL does *not* include the linking restriction. It was originally called the 'Library GPL' (now 'Lesser GPL'), and was designed to allow libraries under a GPL-like licence to be linked to code under other licences.

          Incidentally, Richard Stallman explains on the FSF web site that the idea behind the LGPL is to get programmers hooked on GNU/LGPL libraries, then switch to the full GPL to pull the rug out from under them and force them to either use the GPL for their larger works or go to the considerable trouble of either transitioning to alternative libraries or writing their own. The name change reflects Stallman's belief the GPL is now widespread enough that that it's no longer necessary to encourage libraries to be licensed under the LGPL instead of the full GPL.

  • JFS on Linux (Score:3, Interesting)

    by Nickus ( 10876 ) on Tuesday February 18, 2003 @04:26AM (#5324429)
    I recently tested JFS on a 2TB raid. In two days I got corruption in the filesystem and I was forced to switch back to ext3. This was with the JFS shipped with RH8.
    This doesn't mean anything except I won't touch JFS in a long time. When you need machines in production use you really want to be sure about the filesystems. Too bad the customer wouldn't go with FreeBSD.

  • Hmm (Score:1)

    by cstubbs ( 576917 )
    Oooo, FreeBSD with a decent filesystem finally, after how long ?

    Maybe it will finally be good for something other than routing the odd piece of network traffic about the place.

Slashdot Top Deals

Parts that positively cannot be assembled in improper order will be.

Close