[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: EXT3 with 2.2.20 - Is it stable enough for a production server th at is used 24X7?



On Fri, Nov 16, 2001 at 02:22:31PM +0000, Stephen C. Tweedie wrote:
> 
> Fixing this on 2.2 requires substantial VM surgery and is not a high
> priority.  One thing that could be done relatively simple (and Ted
> Ts'o sent me patches to do this partially) would be to account the
> ext3 VM usage and to throttle ext3 globally if that exceeds a certain
> value.  That is _not_ a real solution: if you do things like
> mlock()ing down some memory, ext3 won't be able to adapt to the new
> free-space availability.  But it would work for most situations.
> 

If you must use 2.2, here are the patches against ext3-0.0.7a.  Note
that there have been further performance (and some bug fixes) made to
the ext3 codebase since the 7a release which are only available with
2.4, so it's highly recommended that you move to 2.4 if you wish to
use ext3.

Still, if you must use 2.2, these patches will allow you to throttle
the number of dirty buffers which ext3 can hold pinned.  You'll have
to adjust /proc/sys/fs/jfs-max-dirty-buffers to taste.... (if you
don't set this sysctl variable, ext3 will use the old behaviour of not
throttling its VM usage).

						- Ted

Patch generated: on Sat Jun  9 12:55:52 EDT 2001 by tytso think
against Linux version 2.2.19
 
===================================================================
RCS file: include/linux/RCS/jfs.h,v
retrieving revision 1.1
diff -u -r1.1 include/linux/jfs.h
--- include/linux/jfs.h	2001/06/09 05:39:04	1.1
+++ include/linux/jfs.h	2001/06/09 15:57:03
@@ -31,6 +31,7 @@
 #endif
 
 extern int journal_enable_debug;
+extern int jfs_max_dirty_buffers, jfs_dirty_buffer_msgs;
 
 #ifdef JFS_DEBUG
 #define jfs_debug(n, f, a...)						\
@@ -261,6 +262,9 @@
 	
 	/* Number of buffers on the t_buffers list */
 	int			t_nr_buffers;
+
+	/* Number of buffers on the t_datalist list */
+	int			t_nr_datalist;
 	
 	/* Doubly-linked circular list of all buffers reserved but not
            yet modified by this transaction */
===================================================================
RCS file: fs/jfs/transaction.c,v
retrieving revision 1.1
diff -u -r1.1 fs/jfs/transaction.c
--- fs/jfs/transaction.c	2001/06/09 05:39:04	1.1
+++ fs/jfs/transaction.c	2001/06/09 05:39:08
@@ -25,6 +25,12 @@
 #include <linux/locks.h>
 #include <linux/timer.h>
 
+/*
+ * Maximum number of dirty data buffers allowed in a running
+ * transaction; zero means unlimited
+ */
+int jfs_max_dirty_buffers;
+int jfs_dirty_buffer_msgs;
 
 /* 
  * Forward declarations for internal use by this file only.
@@ -118,6 +124,7 @@
 {
 	transaction_t *transaction;
 	int needed;
+	int too_many_buffers;
 	int nblocks = handle->h_buffer_credits;
 	
 	jfs_debug(4, "New handle %p going live.\n", handle);
@@ -164,7 +171,16 @@
 
 	needed = transaction->t_outstanding_credits + nblocks;
 
-	if (needed > journal->j_max_transaction_buffers) {
+	too_many_buffers = 0;
+	if (jfs_max_dirty_buffers &&
+	    (transaction->t_nr_datalist > jfs_max_dirty_buffers)) {
+		too_many_buffers = 1;
+		if (jfs_dirty_buffer_msgs)
+			printk("JFS: too many buffers (%d), forcing new transaction.\n", transaction->t_nr_datalist);
+	}
+
+	if ((needed > journal->j_max_transaction_buffers) ||
+	    too_many_buffers) {
 		/* If the current transaction is already too large, then
 		 * start to commit it: we can then go back and attach
 		 * this handle to a new transaction. */
@@ -1110,6 +1126,7 @@
 	case BJ_None:
 		return;
 	case BJ_Data:
+		transaction->t_nr_datalist--;
 		list = &transaction->t_datalist;
 		break;
 	case BJ_Metadata:
@@ -1173,6 +1190,7 @@
 		J_ASSERT(!buf->b_frozen_data);
 		return;
 	case BJ_Data:
+		transaction->t_nr_datalist++;
 		list = &transaction->t_datalist;
 		break;
 	case BJ_Metadata:
===================================================================
RCS file: include/linux/RCS/sysctl.h,v
retrieving revision 1.1
diff -u -r1.1 include/linux/sysctl.h
--- include/linux/sysctl.h	2001/06/09 05:39:04	1.1
+++ include/linux/sysctl.h	2001/06/09 05:39:08
@@ -426,7 +426,10 @@
 	FS_MAXFILE=7,	/* int:maximum number of filedescriptors that can be allocated */
 	FS_DENTRY=8,
 	FS_NRSUPER=9,	/* int:current number of allocated super_blocks */
-	FS_MAXSUPER=10 	/* int:maximum number of super_blocks that can be allocated */
+	FS_MAXSUPER=10,	/* int:maximum number of super_blocks that can be allocated */
+	FS_JFS_DEBUG=100,	/* int: jfs debugging */
+	FS_JFS_MAX_DIRTY_BUFFERS=101, /* int: max # of dirty data buffers */
+	FS_JFS_DIRTY_BUFFER_MSGS=102 /* int: dirty buffer debugging */
 };
 
 /* CTL_DEBUG names: */
===================================================================
RCS file: kernel/RCS/sysctl.c,v
retrieving revision 1.1
diff -u -r1.1 kernel/sysctl.c
--- kernel/sysctl.c	2001/06/09 05:39:04	1.1
+++ kernel/sysctl.c	2001/06/09 05:39:08
@@ -303,7 +303,13 @@
 	{FS_DENTRY, "dentry-state", &dentry_stat, 6*sizeof(int),
 	 0444, NULL, &proc_dointvec},
 #if defined(CONFIG_EXT3_FS) && defined(JFS_DEBUG)
-	{KERN_SHMMAX, "jfs-debug", &journal_enable_debug, sizeof (int),
+	{FS_JFS_DEBUG, "jfs-debug", &journal_enable_debug, sizeof (int),
+	 0644, NULL, &proc_dointvec},
+	{FS_JFS_MAX_DIRTY_BUFFERS, "jfs-max-dirty-buffers",
+	 	 &jfs_max_dirty_buffers, sizeof (int),
+	 0644, NULL, &proc_dointvec},
+	{FS_JFS_DIRTY_BUFFER_MSGS, "jfs-dirty-buffer-msgs",
+	 	 &jfs_dirty_buffer_msgs, sizeof (int),
 	 0644, NULL, &proc_dointvec},
 #endif
 	{0}





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]