PERFORMANCE OF FILESYSTEMS COMPARED (includes Reiser4 and Ext4).

[metala]

Do you know that the developer of ReiserFS (aka Reiser3), Hans Reiser is accused for murdering his wife?

what does this has to do with andrew morton to sabotage the reiser4 fs?! as far as i know andrew is one the people who mostly pushes for reiser4 to being included into the official kernel. if really there was a conspiracy with him involved what reason would he have to continue on this line?!

You didn't understand me ... I thought it will be so.
It was a joke.. lol .. sorry for that, I have an excuse - I was on caffeine.

On the topic: very impressive results.. I still don't know why it's not implemented in kernel mainstream...

[Historian]

Why are comments to

The HANS REISER Murder Trial. Timeline and Analysis

http://phoronix.com/forums/showthread.php?t=7544

thread censored?

Is this usual?

[Michael]

Why are comments to

The HANS REISER Murder Trial. Timeline and Analysis

http://phoronix.com/forums/showthread.php?t=7544

thread censored?

Because you're just trying to spew bullshit that doesn't belong in a technology forum.

[givemesugarr]

You didn't understand me ... I thought it will be so.
It was a joke.. lol .. sorry for that, I have an excuse - I was on caffeine.

On the topic: very impressive results.. I still don't know why it's not implemented in kernel mainstream...

i think that the problem lies in the filesystem corruption. for what i've read on the argument, the standards imposed by linus for the filesystems to be included into the kernel are still not matched by reiser4. andrew morton doesn't think like that and it seems that he is pushing towards its introduction into the main branch, even if with experimental as a feature.

what still leaves me a little puzzled is that ext4 is not so much more stable than reiser4 but it has gotten into the main branch. this only means that the file corruption issue of reiser4 is really bad when compared to the one of ext4.

[rbmorse]

Because you're just trying to spew bullshit that doesn't belong in a technology forum.

That's a real good reason. Michael, please continue to enforce that policy with rigor and enthusiasm.

[Jade]

The Linux Kernel SABOTEURS @ http://www.phoronix.com/forums/showthread.php?t=9509

The Linux Kernel Saboteurs.

From http://linuxhelp.150m.com/
and http://linux.50webs.org

Although the Linux kernel saboteurs (a number of so called Linux kernel "developers") have sabotaged many areas of Linux, I will concentrate on their efforts in sabotaging Reiser4.

As some will have noticed, Reiser4 no longer works as it used to. The recent patches of Andrew Morton, and Riffard Laurent, cause corruption problems (when they work at all). Morton's patches, also reduced Reiser4's functionality, removing, for example, the cryptocompress plugin. Riffard's patches include the cryptocompress plugin, which seems to work, but the patch causes corruption for plain Reiser4.

You see, the Reiser4 saboteurs arranged that plain Reiser4 will not work properly. However, they forgot to sabotage the more complex case of transparent compression, so we have the weird situation where the more complex case works fine, while the kernel "developers" "struggle" to get the simple case to work like it used to.

The parts of Reiser4 they have not touched, still work, but where they have coded, Reiser4 no longer works.

The fact that Reiser4 worked well, before Hans Reiser's arrest and imprisonment (on what appears to be trumped-up charges) and now doesn't, means that it should be easy to spot the sabotage.

After digging around in the source code, the evidence of deliberate sabotage is very clear.

I present some of the evidence (found by comparing Riffard's patch with older Namesys patches) below.

Update: Namesys has released 2.6.20 and 2.6.21 kernel patches. This has enabled one to completely isolate the differences between Namesys' 2.6.20 patch and Riffard's. The diff file is very small and can be found here. The early sabotage is concentrated in this small file.

One act of sabotage, is found in reiser4/page_cache.c:

int reiser4_page_io(struct page *page, jnode *node, int rw, gfp_t gfp)
{
	struct bio *bio;
	int result;

	assert("nikita-2094", page != NULL);
	assert("nikita-2226", PageLocked(page));
	assert("nikita-2634", node != NULL);
	assert("nikita-2893", rw == READ || rw == WRITE);

	if (rw) {
		if (unlikely(page->mapping->host->i_sb->s_flags & MS_RDONLY)) {
			unlock_page(page);
			return 0;
		}
	}

	bio = page_bio(page, node, rw, gfp);
	if (!IS_ERR(bio)) {
		if (rw == WRITE) {
			SetPageWriteback(page); 
	[color=black]Changed to[/color]	[color=red]set_page_writeback(page);[/color]
			unlock_page(page);
		}
		reiser4_submit_bio(rw, bio);
		result = 0;
	} else {
		unlock_page(page);
		result = PTR_ERR(bio);
	}

	return result;
}

where SetPageWriteback has been changed to set_page_writeback.

This change is hard to spot with SetPageWriteback and set_page_writeback, being very similar in name. This change is almost guaranteed to cause problems, as set_page_writeback is called without calling end_page_writeback(). According to Documentation/filesystems/Locking, this will leave the page itself marked clean but it will be tagged as dirty in the radix tree. This incoherency can lead to all sorts of hard-to-debug problems in the filesystem like having dirty inodes at umount and losing written data. Just as the saboteurs desire, and what we see happening.

The definition of SetPageWriteback is found in linux-2.6.20/include/linux/page-flags.h,

#define SetPageWriteback(page)                                          \
        do {                                                            \
                if (!test_and_set_bit(PG_writeback,                     \
                                &(page)->flags))                        \
                        inc_zone_page_state(page, NR_WRITEBACK);        \
        } while (0)
as is the definition of [color=red]set_page_writeback,[/color]
static inline void set_page_writeback(struct page *page)
{
        test_set_page_writeback(page);
}

where test_set_page_writeback (from linux-2.6.20/mm/page-writeback.c) is:

int test_set_page_writeback(struct page *page)
{
        struct address_space *mapping = page_mapping(page);
        int ret;

        if (mapping) {
                unsigned long flags;

                write_lock_irqsave(&mapping->tree_lock, flags);
                ret = TestSetPageWriteback(page);
                if (!ret)
                        radix_tree_tag_set(&mapping->page_tree,
                                                page_index(page),
                                                PAGECACHE_TAG_WRITEBACK);
                if (!PageDirty(page))
                        radix_tree_tag_clear(&mapping->page_tree,
                                                page_index(page),
                                                PAGECACHE_TAG_DIRTY);
                write_unlock_irqrestore(&mapping->tree_lock, flags);
        } else {
                ret = TestSetPageWriteback(page);
        }
        return ret;

}
EXPORT_SYMBOL(test_set_page_writeback);

From the above definitions, it is clear that swapping set_page_writeback for SetPageWriteback was not an accidental mistake (the code is completely different).

The patch, move-page-writeback-acounting-out-of-macros.patch, is Andrew Morton's attempt to completely cover his tracks. He claims it is "inconsistent" and "awkward" to have the three page-writeback accounting macros in include/linux/page-flags.h (together with all the other page macros) and that it would be better somehow, if they were moved to mm/page-writeback.c. However, not a single one of these definitions is actually transfered to mm/page-writeback.c. Two of them have their definition split between the two named files and the third, SetPageWriteback, is "accidentally" lost. It just disappears.

In this way, Andrew Morton completely eliminates all reference to SetPageWriteback in the (2.6.21) mm-kernel.

The patch move-page-writeback-acounting-out-of-macros.patch is presented here:
From: Andrew Morton

page-writeback accounting is presently performed in the page-flags macros.
This is inconsistent and makes it awkward to implement per-backing_dev
under-writeback page accounting.

So move this accounting down to the callsite(s).

Signed-off-by: Andrew Morton 
---

 include/linux/page-flags.h |   38 +++++++----------------------------
 mm/page-writeback.c        |    4 +++
 2 files changed, 12 insertions(+), 30 deletions(-)

diff -puN include/linux/page-flags.h~move-page-writeback-acounting-out-of-macros include/linux/page-flags.h
--- a/include/linux/page-flags.h~move-page-writeback-acounting-out-of-macros
+++ a/include/linux/page-flags.h
@@ -184,37 +184,15 @@ static inline void SetPageUptodate(struc
 #define __SetPagePrivate(page)  __set_bit(PG_private, &(page)->flags)
 #define __ClearPagePrivate(page) __clear_bit(PG_private, &(page)->flags)
 
+/*
+ * Only test-and-set exist for PG_writeback.  The unconditional operators are
+ * risky: they bypass page accounting.
+ */
 #define PageWriteback(page)	test_bit(PG_writeback, &(page)->flags)
-#define SetPageWriteback(page)						\
-	do {								\
-		if (!test_and_set_bit(PG_writeback,			\
-				&(page)->flags))			\
-			inc_zone_page_state(page, NR_WRITEBACK);	\
-	} while (0)
-#define TestSetPageWriteback(page)					\
-	({								\
-		int ret;						\
-		ret = test_and_set_bit(PG_writeback,			\
-					&(page)->flags);		\
-		if (!ret)						\
-			inc_zone_page_state(page, NR_WRITEBACK);	\
-		ret;							\
-	})
-#define ClearPageWriteback(page)					\
-	do {								\
-		if (test_and_clear_bit(PG_writeback,			\
-				&(page)->flags))			\
-			dec_zone_page_state(page, NR_WRITEBACK);	\
-	} while (0)
-#define TestClearPageWriteback(page)					\
-	({								\
-		int ret;						\
-		ret = test_and_clear_bit(PG_writeback,			\
-				&(page)->flags);			\
-		if (ret)						\
-			dec_zone_page_state(page, NR_WRITEBACK);	\
-		ret;							\
-	})
+#define TestSetPageWriteback(page) test_and_set_bit(PG_writeback,	\
+							&(page)->flags)
+#define TestClearPageWriteback(page) test_and_clear_bit(PG_writeback,	\
+							&(page)->flags)
 
 #define PageBuddy(page)		test_bit(PG_buddy, &(page)->flags)
 #define __SetPageBuddy(page)	__set_bit(PG_buddy, &(page)->flags)
diff -puN mm/page-writeback.c~move-page-writeback-acounting-out-of-macros mm/page-writeback.c
--- a/mm/page-writeback.c~move-page-writeback-acounting-out-of-macros
+++ a/mm/page-writeback.c
@@ -987,6 +987,8 @@ int test_clear_page_writeback(struct pag
 	} else {
 		ret = TestClearPageWriteback(page);
 	}
+	if (ret)
+		dec_zone_page_state(page, NR_WRITEBACK);
 	return ret;
 }
 
@@ -1012,6 +1014,8 @@ int test_set_page_writeback(struct page 
 	} else {
 		ret = TestSetPageWriteback(page);
 	}
+	if (!ret)
+		inc_zone_page_state(page, NR_WRITEBACK);
 	return ret;
 
 }
_

Another, hard to spot act of sabotage, occurs in reiser4/jnode.c, where the order of 2 lines has been swapped. This:

[givemesugarr]

Morton's patches, also reduced Reiser4's functionality, removing, for example, the cryptocompress plugin. Riffard's patches include the cryptocompress plugin, which seems to work, but the patch causes corruption for plain Reiser4.

did that work?! if it didn't work and it didn't work?! also if the kernel has its own crypto plugin why the hell does a filesystem in the kernel need it?! it will use the kernel api for this.

You see, the Reiser4 saboteurs arranged that plain Reiser4 will not work properly. However, they forgot to sabotage the more complex case of transparent compression, so we have the weird situation where the more complex case works fine, while the kernel "developers" "struggle" to get the simple case to work like it used to.

hmmm... an example would be better.

The parts of Reiser4 they have not touched, still work, but where they have coded, Reiser4 no longer works.

what are these parts?! what are the parts touched that don't work anymore?!

The fact that Reiser4 worked well, before Hans Reiser's arrest and imprisonment (on what appears to be trumped-up charges) and now doesn't, means that it should be easy to spot the sabotage.

again: where's the proof that it worked fine before?!

After digging around in the source code, the evidence of deliberate sabotage is very clear.

I present some of the evidence (found by comparing Riffard's patch with older Namesys patches) below.

Update: Namesys has released 2.6.20 and 2.6.21 kernel patches. This has enabled one to completely isolate the differences between Namesys' 2.6.20 patch and Riffard's. The diff file is very small and can be found here. The early sabotage is concentrated in this small file.

One act of sabotage, is found in reiser4/page_cache.c:

where SetPageWriteback has been changed to set_page_writeback.

This change is hard to spot with SetPageWriteback and set_page_writeback, being very similar in name. This change is almost guaranteed to cause problems, as set_page_writeback is called without calling end_page_writeback(). According to Documentation/filesystems/Locking, this will leave the page itself marked clean but it will be tagged as dirty in the radix tree. This incoherency can lead to all sorts of hard-to-debug problems in the filesystem like having dirty inodes at umount and losing written data. Just as the saboteurs desire, and what we see happening.

The definition of SetPageWriteback is found in linux-2.6.20/include/linux/page-flags.h,

#define SetPageWriteback(page)                                          \
        do {                                                            \
                if (!test_and_set_bit(PG_writeback,                     \
                                &(page)->flags))                        \
                        inc_zone_page_state(page, NR_WRITEBACK);        \
        } while (0)
as is the definition of [color=red]set_page_writeback,[/color]
static inline void set_page_writeback(struct page *page)
{
        test_set_page_writeback(page);
}

where test_set_page_writeback (from linux-2.6.20/mm/page-writeback.c) is:

int test_set_page_writeback(struct page *page)
{
        struct address_space *mapping = page_mapping(page);
        int ret;

        if (mapping) {
                unsigned long flags;

                write_lock_irqsave(&mapping->tree_lock, flags);
                ret = TestSetPageWriteback(page);
                if (!ret)
                        radix_tree_tag_set(&mapping->page_tree,
                                                page_index(page),
                                                PAGECACHE_TAG_WRITEBACK);
                if (!PageDirty(page))
                        radix_tree_tag_clear(&mapping->page_tree,
                                                page_index(page),
                                                PAGECACHE_TAG_DIRTY);
                write_unlock_irqrestore(&mapping->tree_lock, flags);
        } else {
                ret = TestSetPageWriteback(page);
        }
        return ret;

}
EXPORT_SYMBOL(test_set_page_writeback);

From the above definitions, it is clear that swapping set_page_writeback for SetPageWriteback was not an accidental mistake (the code is completely different).

IT'S CLEAR THAT THE CODE IS DIFFERENT, BUT IT'S NOT EVEN 1% CLEAR THAT THE NEW CODE WOULD WORK WORSE. YOU NEED TO EXPLICIT EVERY SINGLE FUNCTION USED IN THIS CODE WITH EVERYTHING THAT IT DOES TO BE ABLE TO SAY THAT THE MODIFIED CODE WORK WORSE. NOW, BEFORE THIS DOCUMENTATION IS SHOWN I'LL CONTINUE TO THINK THAT THE NEW CODE IS BETTER THAT THE OLD ONE.

The patch, move-page-writeback-acounting-out-of-macros.patch, is Andrew Morton's attempt to completely cover his tracks. He claims it is "inconsistent" and "awkward" to have the three page-writeback accounting macros in include/linux/page-flags.h (together with all the other page macros) and that it would be better somehow, if they were moved to mm/page-writeback.c. However, not a single one of these definitions is actually transfered to mm/page-writeback.c. Two of them have their definition split between the two named files and the third, SetPageWriteback, is "accidentally" lost. It just disappears.

HE'S DAMN RIGHT AND USUALLY PROGRAMMERS KNOW THIS TOO.

In this way, Andrew Morton completely eliminates all reference to SetPageWriteback in the (2.6.21) mm-kernel.

The patch move-page-writeback-acounting-out-of-macros.patch is presented here:
From: Andrew Morton

page-writeback accounting is presently performed in the page-flags macros.
This is inconsistent and makes it awkward to implement per-backing_dev
under-writeback page accounting.

So move this accounting down to the callsite(s).

Signed-off-by: Andrew Morton 
---

 include/linux/page-flags.h |   38 +++++++----------------------------
 mm/page-writeback.c        |    4 +++
 2 files changed, 12 insertions(+), 30 deletions(-)

diff -puN include/linux/page-flags.h~move-page-writeback-acounting-out-of-macros include/linux/page-flags.h
--- a/include/linux/page-flags.h~move-page-writeback-acounting-out-of-macros
+++ a/include/linux/page-flags.h
@@ -184,37 +184,15 @@ static inline void SetPageUptodate(struc
 #define __SetPagePrivate(page)  __set_bit(PG_private, &(page)->flags)
 #define __ClearPagePrivate(page) __clear_bit(PG_private, &(page)->flags)
 
+/*
+ * Only test-and-set exist for PG_writeback.  The unconditional operators are
+ * risky: they bypass page accounting.
+ */
 #define PageWriteback(page)	test_bit(PG_writeback, &(page)->flags)
-#define SetPageWriteback(page)						\
-	do {								\
-		if (!test_and_set_bit(PG_writeback,			\
-				&(page)->flags))			\
-			inc_zone_page_state(page, NR_WRITEBACK);	\
-	} while (0)
-#define TestSetPageWriteback(page)					\
-	({								\
-		int ret;						\
-		ret = test_and_set_bit(PG_writeback,			\
-					&(page)->flags);		\
-		if (!ret)						\
-			inc_zone_page_state(page, NR_WRITEBACK);	\
-		ret;							\
-	})
-#define ClearPageWriteback(page)					\
-	do {								\
-		if (test_and_clear_bit(PG_writeback,			\
-				&(page)->flags))			\
-			dec_zone_page_state(page, NR_WRITEBACK);	\
-	} while (0)
-#define TestClearPageWriteback(page)					\
-	({								\
-		int ret;						\
-		ret = test_and_clear_bit(PG_writeback,			\
-				&(page)->flags);			\
-		if (ret)						\
-			dec_zone_page_state(page, NR_WRITEBACK);	\
-		ret;							\
-	})
+#define TestSetPageWriteback(page) test_and_set_bit(PG_writeback,	\
+							&(page)->flags)
+#define TestClearPageWriteback(page) test_and_clear_bit(PG_writeback,	\
+							&(page)->flags)
 
 #define PageBuddy(page)		test_bit(PG_buddy, &(page)->flags)
 #define __SetPageBuddy(page)	__set_bit(PG_buddy, &(page)->flags)
diff -puN mm/page-writeback.c~move-page-writeback-acounting-out-of-macros mm/page-writeback.c
--- a/mm/page-writeback.c~move-page-writeback-acounting-out-of-macros
+++ a/mm/page-writeback.c
@@ -987,6 +987,8 @@ int test_clear_page_writeback(struct pag
 	} else {
 		ret = TestClearPageWriteback(page);
 	}
+	if (ret)
+		dec_zone_page_state(page, NR_WRITEBACK);
 	return ret;
 }
 
@@ -1012,6 +1014,8 @@ int test_set_page_writeback(struct page 
 	} else {
 		ret = TestSetPageWriteback(page);
 	}
+	if (!ret)
+		inc_zone_page_state(page, NR_WRITEBACK);
 	return ret;
 
 }
_

Another, hard to spot act of sabotage, occurs in reiser4/jnode.c, where the order of 2 lines has been swapped. This:

[/quote]

I TEND TO AGREE WITH MORTON'S OBSERVATIONS AND TO BE ON HIS SIDE THIS TIME. IF SOMEONE WHO "CLAIMS" TO KNOW SO MUCH REISER4 IS THERE WHY THE HE** HE DOESN'T CONTACT MORTON AND HELP HIM WITH PUTTING REISER4 INTO THE KERNEL?! I'LL TELL YOU WHY:
BECAUSE IT'S EASIER TO SEE CONSPIRACIES AND TO SHOUT THEM OUT LOUD THAN REALLY DOING SOMETHING.

[hoho]

Just don't feed the trolls.

[Jade]

You see, the Reiser4 saboteurs arranged that plain Reiser4 will not work properly. However, they forgot to sabotage the more complex case of transparent compression, so we have the weird situation where the more complex case works fine, while the kernel "developers" "struggle" to get the simple case to work like it used to.

This is an excellent point.

[lenrek]

Just don't feed the trolls.

Hey, what is the fun then?