[PATCH] kthread primitive

From: Rusty Russell <rusty@rustcorp.com.au>

These two patches provide the framework for stopping kernel threads to
allow hotplug CPU.  This one just adds kthread.c and kthread.h, next
one uses it.

Most importantly, adds a Monty Python quote to the kernel.

Details:

The hotplug CPU code introduces two major problems:

1) Threads which previously never stopped (migration thread,
   ksoftirqd, keventd) have to be stopped cleanly as CPUs go offline.
2) Threads which previously never had to be created now have
   to be created when a CPU goes online.

Unfortunately, stopping a thread is fairly baroque, involving memory
barriers, a completion and spinning until the task is actually dead
(for example, complete_and_exit() must be used if inside a module).

There are also three problems in starting a thread:
1) Doing it from a random process context risks environment contamination:
   better to do it from keventd to guarantee a clean environment, a-la
   call_usermodehelper.
2) Getting the task struct without races is a hard: see kernel/sched.c
   migration_call(), kernel/workqueue.c create_workqueue_thread().
3) There are races in starting a thread for a CPU which is not yet
   online: migration thread does a complex dance at the moment for
   a similar reason (there may be no migration thread to migrate us).

Place all this logic in some primitives to make life easier:
kthread_create() and kthread_stop().  These primitives require no
extra data-structures in the caller: they operate on normal "struct
task_struct"s.

Other changes:

- Expose keventd_up(), as keventd and migration threads will use kthread to
  launch, and kthread normally uses workqueues and must recognize this case.

- Kthreads created at boot before "keventd" are spawned directly.  However,
  this means that they don't have all signals blocked, and hence can be
  killed.  The simplest solution is to always explicitly block all signals in
  the kthread.

- Change over the migration threads, the workqueue threads and the
  ksoftirqd threads to use kthread.

- module.c currently spawns threads directly to stop the machine, so a
  module can be atomically tested for removal.

- Unfortunately, this means that the current task is manipulated (which
  races with set_cpus_allowed, for example), and it can't set its priority
  artificially high.  Using a kernel thread can solve this cleanly, and with
  kthread_run, it's simple.

- kthreads use keventd, so they inherit its cpus_allowed mask.  Unset it.
  All current users set it explicity anyway, but it's nice to fix.

- call_usermode_helper uses keventd, so the process created inherits its
  cpus_allowed mask.  Unset it.

- Prevent errors in boot when cpus_possible() contains a cpu which is not
  online (ie.  a cpu didn't come up).  This doesn't happen on x86, since a
  boot failure makes that CPU no longer possible (hacky, but it works).

- When the cpu fails to come up, some callbacks do kthread_stop(), which
  doesn't work without keventd (which hasn't started yet).  Call it directly,
  and take care that it restores signal state (note: do_sigaction does a
  flush on blocked signals, so we don't need to repeat it).

1 files changed, 81 insertions, 0 deletions

diff --git a/include/linux/kthread.h b/include/linux/kthread.h
new file mode 100644
index 000000000000..3fa786448db3
--- /dev/null
+++ b/include/linux/kthread.h
@@ -0,0 +1,81 @@
+#ifndef _LINUX_KTHREAD_H
+#define _LINUX_KTHREAD_H
+/* Simple interface for creating and stopping kernel threads without mess. */
+#include <linux/err.h>
+#include <linux/sched.h>
+
+/**
+ * kthread_create: create a kthread.
+ * @threadfn: the function to run until signal_pending(current).
+ * @data: data ptr for @threadfn.
+ * @namefmt: printf-style name for the thread.
+ *
+ * Description: This helper function creates and names a kernel
+ * thread.  The thread will be stopped: use wake_up_process() to start
+ * it.  See also kthread_run(), kthread_create_on_cpu().
+ *
+ * When woken, the thread will run @threadfn() with @data as its
+ * argument. @threadfn can either call do_exit() directly if it is a
+ * standalone thread for which noone will call kthread_stop(), or
+ * return when 'kthread_should_stop()' is true (which means
+ * kthread_stop() has been called).  The return value should be zero
+ * or a negative error number: it will be passed to kthread_stop().
+ *
+ * Returns a task_struct or ERR_PTR(-ENOMEM).
+ */
+struct task_struct *kthread_create(int (*threadfn)(void *data),
+				   void *data,
+				   const char namefmt[], ...);
+
+/**
+ * kthread_run: create and wake a thread.
+ * @threadfn: the function to run until signal_pending(current).
+ * @data: data ptr for @threadfn.
+ * @namefmt: printf-style name for the thread.
+ *
+ * Description: Convenient wrapper for kthread_create() followed by
+ * wake_up_process().  Returns the kthread, or ERR_PTR(-ENOMEM). */
+#define kthread_run(threadfn, data, namefmt, ...)			   \
+({									   \
+	struct task_struct *__k						   \
+		= kthread_create(threadfn, data, namefmt, ## __VA_ARGS__); \
+	if (!IS_ERR(__k))						   \
+		wake_up_process(__k);					   \
+	__k;								   \
+})
+
+/**
+ * kthread_bind: bind a just-created kthread to a cpu.
+ * @k: thread created by kthread_create().
+ * @cpu: cpu (might not be online, must be possible) for @k to run on.
+ *
+ * Description: This function is equivalent to set_cpus_allowed(),
+ * except that @cpu doesn't need to be online, and the thread must be
+ * stopped (ie. just returned from kthread_create().
+ */
+void kthread_bind(struct task_struct *k, unsigned int cpu);
+
+/**
+ * kthread_stop: stop a thread created by kthread_create().
+ * @k: thread created by kthread_create().
+ *
+ * Sets kthread_should_stop() for @k to return true, wakes it, and
+ * waits for it to exit.  Your threadfn() must not call do_exit()
+ * itself if you use this function!  This can also be called after
+ * kthread_create() instead of calling wake_up_process(): the thread
+ * will exit without calling threadfn().
+ *
+ * Returns the result of threadfn(), or -EINTR if wake_up_process()
+ * was never called. */
+int kthread_stop(struct task_struct *k);
+
+/**
+ * kthread_should_stop: should this kthread return now?
+ *
+ * When someone calls kthread_stop on your kthread, it will be woken
+ * and this will return true.  You should then return, and your return
+ * value will be passed through to kthread_stop().
+ */
+int kthread_should_stop(void);
+
+#endif /* _LINUX_KTHREAD_H */