nVidia 325.15+Linux-rt: old && new nvidia-rt patch/methods; Questions about nvidia-installer

Hi nvidia developers.

I have been able to get the 325.15 driver succussfully working with linux-rt (3.10.x-rt series), but am still looking for information / get a question or two ansered, as well as providing some info / patches too…

It seems that the 325.xx series nvidia installer does not need patching, in regards to linux-rt (or rather “less patching”- as the old nvidia-rt patch that adds some spinlock related stuff, isn’t needed; it’s included in 325.15). For brief background, previously we (linux-rt/nvidia users) used to have to patch nv-linux.h to support CONFIG_PREEMPT_RT_FULL + add missing bits for RT.

  • Now that nvidia has added the missing stuff needed by -rt kernels, I see that we can also override the PREEMPT_RT check/behavior <in nvidia-insaller/conftest> by use of IGNORE_PREEMPT_RT_PRESENCE=1 environment variable… → but is this correct?..or do we need to do something else for the installer (and driver) to work for -rt??. —>(as i have done below)<-- i wasn’t able to find any documentation on the matter…

That being said i still did end up having to patch nv-linux.h. like so;


— a/nv-linux.h 2011-10-26 13:35:32.866579965 +0200
+++ b/nv-linux.h 2011-10-26 13:35:47.265117607 +0200
@@ -43,6 +43,8 @@
#include <linux/version.h>
#include <linux/utsname.h>

+#define CONFIG_PREEMPT_RT_FULL 1
+
#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 9)

error This driver does not support kernels older than 2.6.9!

#elif LINUX_VERSION_CODE < KERNEL_VERSION(2, 7, 0)
@@ -312,11 +312,7 @@
#endif
#endif

-#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_PREEMPT_RT_FULL)
-#define NV_CONFIG_PREEMPT_RT 1
-#endif

-#if defined(NV_CONFIG_PREEMPT_RT)
+#if defined(CONFIG_PREEMPT_RT_FULL)
typedef raw_spinlock_t nv_spinlock_t;
#define NV_SPIN_LOCK_INIT(lock) raw_spin_lock_init(lock)
#define NV_SPIN_LOCK_IRQ(lock) raw_spin_lock_irq(lock)
@@ -943,7 +939,7 @@
return ret;
}

-#if defined(NV_CONFIG_PREEMPT_RT)
+#if defined(CONFIG_PREEMPT_RT_FULL)
#define NV_INIT_MUTEX(mutex) sema_init(mutex,1)
#else
#if !defined(__SEMAPHORE_INITIALIZER) && defined(__COMPAT_SEMAPHORE_INITIALIZER)


…basically, I am explicitly (and/or redundantly?) defining/using CONFIG_PREEMPT_RT_FULL (while also disabling the PREEMPT_RT conftest). Using this patch AND disabling the PREEMPT_RT conftest together, yields a driver that works… However, i was under the impression that disabling the check was enough(?)

  • this morning i discovered that by NOT using the above patch (because i thought it wasn’t needed); while i was able to compile the nvidia driver (since i am disabling the check) → the resulting driver locked up my machine(s). → This would indicate to me that either; A) there is a way to tell the nvidia installer to compile for RT <since support appears to be there / disabled by default> OR B). Linux RT users are still required to patch the driver and disable the check.

note: the patched-version (of 325.15) works great / as expected.

anyway, it would be great to have some clarification from an nvidia-developer on the matter, as it effects users, packagers, etc like myself. At this point, it seems that the 325.xx series has changed for us, but there isn’t anything really documented, which would be helpful. :) ~ but again, i do have it working (after some tinkering). My uptime has been over 2 days with both 3.10.4-rt1 and 3.10.6-rt3 <total of 4 dayz X2 machines, no issues> with 9800GT and newer 440GT.

thx

Just in case, any user/packager who reads this and is looking for nvidia-rt related patches, i have patches for nvidia driver + linux-3.10.x-rt;

  • you basically, run the NVIDIA_XXXxxx.run file with “–extract-only” option
  • then; cd into /home/USERNAME/Desktop/NVIDIA-Linux-x86_64-325.15-no-compat32/kernel
  • apply patch; " patch -Np1 -i nvidia-325xx-rt.patch "
  • run the installer, compile && install driver, etc, as you normally would (or make your own buildscript, source package, etc)

1.) 319.32 → patches for nvidia-rt and linux 3.10

a. legacy (nvidia) RT patch: Download L_Pa Project from SourceForge.net
b. linux 3.10 / 319.32 patch: Download L_Pa Project from SourceForge.net

2). 325.15 → patch for nvidia-325.xx-rt + IGNORE_PREEMPT_RT_PRESENCE=1

a. 325.xx (nvidia) RT patch: Download L_Pa Project from SourceForge.net
b. with 325.15 you must also override the PREEMPT_RT conftest, like so (when building);

" make IGNORE_PREEMPT_RT_PRESENCE=1 SYSSRC=/usr/lib/modules/“${_kernver}/build” module "

obviously, you must have the corresponding / matching nvidia-utils package(s) installed, as well / however your distribution’s package management handles this. <i use Archlinux, thus have “source packages” / PKGUBILDs aka: build scripts for this purpose.>

note: the legacy patch (#1/a) can be used on any nvidia driver pre-325.xx series, during linux(-rt) kernel 3.x development (but not 2.6-rt kernels)… the legacy nvidia-rt patch isn’t needed/applicable by 325.xx, since most of those changes have been integrated into that series/driver version.

I guess for now, for my own packages, i will just use #2 method/version (325.15) But i would still like to know if there is a proper way with 325.15 to build for RT without patching? … I’m also curious if nvidia is working towards supporting linux-rt “offically” one day?

Three days/ 100+ views and no ‘takers’ eh?

Well, just as an update on the matter. I have spoken to someone else who did the same comparison of; overriding the PREEMPT_RT check without the nvidia-325xx-rt.patch VS. using the override with nvidia-325xx-rt.patch … He yielded similar results, as i did… So disabling the check does not appear to be enough… but for the both of us, the latest nvidia(-rt) driver is working.

I for one would also be very happy for this to be fixed. I need to use the realtime patch kernel for low latency audio, read Digital Audio Workstation (and other music oriented software). On older kernels and Nvidia drivers I got mostly very good performance out of the Nvidia driver once patched. There were only a few specific actions that would cause an interruption in the audio streaming, like switching virtual terminals, starting video playback, etc. None of them something that I’d normally do while recording audio, so merely annoyances. Nowdays I can’t use the Nvidia driver anymore, and use the Intel HD3000 on my i7-2600k with excellent results, drawback being less quality in video playback and low performance when playing the occasional game.

@Jhernberg: Thanks for posting / letting us know, here in the forum.

I think it would be nice to just have an option when you run the installer (like adding --preepmt-rt to the commandline).


***Just as a general note: I’ve been noticing downloads on my SF.net account (from the links found here), so i would be curious to know if any users in the forums have any feedback, thoughts, etc on using the nvidia driver with linux-rt??

cheerz

Hello, new user to the forum, got here when I was looking for a solution to the nvidia driver complaining about my RT-kernel.

I use kernel 3.10.6 patched with OSADL patch rt3
nvidia driver 325.15
I’ve patched the driver, but the installer still complains, i guess I must rebuild it to?
The problem is I don’t know exactly how, I’m running debian squeeze and can’t find any package or source anywhere to build the installer, could someone maybe point me in the right direction or maybe just supply a patched and compiled installer that ignores the RT-flag in the kernel config?

Hey ulflj,

Unfortunately, i do not use Debian, so i personally can’t recommend ‘debian specific’ instructions for you. However, google turned up kernel source packages for 325.15 nvidia driver; Debian -- Details of package nvidia-kernel-source in experimental

There appears to be a readme for debian installation, as well. I’m not sure how to add a patch to a deb to use in builds, but i am sure there are instructions and/or examples of other packages kicking around the web (or source packages found in your repos, if i remember correctly; apt allows you to donwload source packages for modification/rebuilding…).

as for ignoring the PREEMPT_RT_FULL flag; I’ve already given you instructions on how to do that. On installation; ie: running the “make” command, part of the installation * you must add IGNORE_PREEMPT_RT_PRESENCE=1 to the commandline <read post #2 again>.

anyway, you’ll have to read up on how debian does things, but aside from that; as long as you’ve patched 325.15 with nvidia-325xx-rt.patch && used IGNORE_PREEMPT_RT_PRESENCE=1 together. it should work. I’ve had a couple people report back to me via email on it (one Ubuntu, one Fedora)… <and obviously I am using my own source packages for Archlinux on 2 machines>

in all cases that i am aware of; IGNORE_PREEMPT_RT_PRESENCE=1 on it’s own will allow the driver to compile, but doesn’t seem to be enough for PREEMPT_RT_FULL, but doing both (patching / override RT check) seems to work for those whom know their H/W works with RT.

@ulflj

I didn’t have time earlier, but just now took a look at the nvidia-kernel-source (325.15) for debian.

I extracted/unarchived the .deb, followed by data.tar.xz and then finally nvidia-kernel.tar.xz (which is inside of data.tar.xz). It appears debian already has a queue of patches in their nvidia source package;

…/nvidia-kernel/debian/patches

I think the tool you will need to modify the package (adding the nvidia-325xx-rt.patch) is called “dpatch”. Just search/google “dpatch tutorial”, there are a lot of tutorials (you can probably find a better one than I, as you are more familiar with debian), so you should be able to figure it out.

dpatch is used to manage those patches in debian/patches, as well as making other adjustments to the .deb file / build instructions… After you’ve gotten this to work, next you will need to figure out at what step / where does debian execute the “make” command for the nvidia driver; at that point you will need to add the IGNORE_PREEMPT_RT_PRESENCE=1 to the commandline, to ensure the nvidia-installer overrides the PREEMPT_RT test. then you should be home free. This may be handled in the .deb or possibly another tool used in debian (like a module helper script or something like that)… this i don’t know, but i am not installing debian to find out ;)

Maybe another Debian user, more familiar with debian packaging will stumble on your post (or try debian forums, maybe ask for a pakcager’s insights / help on how to proceed.)… Regardless, do keep me posted on how things work out, k?

Thank you very much for the help, greatly apriciated, i’m not very familiar with more advanced make-processes than building/linking simple c-programs though, so i think I need help to sort out exactly what I need to change/build.

  1. I get the NVIDIA-Linux-x86-325.15.run from the nvidia site.
  2. I do NVIDIA-Linux-x86-325.15.run --extract-only to decompsress the driver/installer
  3. I download the nvidia-325xx-rt.patch
  4. I cd to NVIDIA-Linux-x86-325.15/kernel/ and do the patch with patch -Np1 -i nvidia-325xx-rt.patch
  5. If I now run the nvidia-installer (the one in NVIDIA-Linux-x86-325.15 dir) it halts because “ERROR: The kernel you are installing for is a PREEMPT_RT kernel!”

So, what I need to do is rebuild the nvidia-installer to ignore that my kernel is preempt_rt, or have I misunderstood everything?

(to clarify, i use a vanilla kernel from kernel.org wich i patched with a rt-patched, not an native debian-kernel)
(clarify 2, the NVIDIA-Linux-x86-325.15.run installs and work ok before I apply the rt-patch)

Ok, another try.
downloaded Debian -- Details of package nvidia-kernel-source in experimental package
Installed it with dpkg
cd /usr/src, untar nvidia-kernel.tar.xz that the package placed there (creates /usr/src/modules)
cd /usr/src/modules/nvidia-kernel patch nv-linux.h with “patch -Np1 -i nvidia-325xx-rt.patch”
read /usr/share/doc/nvidia-kernel-source/README.Debian.gz (method #4)
“apt-get install kernel-package”
“apt-get install nvidia-kernel-common”
cd /usr/src/linux-3.10.6-rt3
“make-kpkg modules_image” (builds package with nvidia kernel module and places it in /usr/src)
Extract kernel module from that package with “dpkg --fsys-tarfile nvidia-kernel-3.10.6-rt3_325.15-1+3.10.6-rt3-10.00.Custom_i386.deb | tar xOf - ./lib/modules/3.10.6-rt3/nvidia/nvidia-current.ko >nvidia.ko” (package had unmet deps that I couldn’t resolv)
copy nvidia.ko to your default module location, in debian; /lib/modules/3.10.6/kernel/drivers/video/nvidia.ko
seems to work (no extensive rt-testing done yet, will try to report back when I’ve done that.)

So if you want the nvidia tools (ie nvdia-settings) install the original driver without an rt-kernel, then compile the driver separatley from standalone package.

@ulflj

I can’t say much about the unmet dependencies and such, or even much about your building method. But i am still wondering, if it wouldn’t be easier to just adjust the source package (since it already does all of these steps?)… that is assuming you could work out the unmet dependencies… I only ask; because in the future aren’t you just going to run into this again and have to manually resolve everything again??

food for thought, i suppose…

So to clarify, you have linux-3.10.6-rt3 + 525.15 (nvidia-rt) working successfully. ie: you’ve booted into it, no lockups or anything like that?? (even if it took a little extra tweaking / manual intervention?) If so; Awesome asnd i am glad i was helpful, even though i am not a debian user ;) … Out of curiousity; would you care to share you setup? For example, my system (that i am typing from is);

  • AMD Phenom II 965 black edition (3.4GHZ X 4 cores)
  • 16GIG DDR3 RAM
  • 2TB SATA HDD
  • ASUS MOBO (M4N75TD / Xtreme Design)
  • nVidia 440GT

Running Archlinux 64bit(Multilib).

$ uname -a
Linux localhost.localdomain 3.10.6-rt3-1-l-pa #1 SMP PREEMPT RT Thu Aug 22 21:00:54 EDT 2013 x86_64 GNU/Linux

cheerz.

Well maybe, I hopefully won’t be recompiling the driver very often, and I made a script that does all the steps and compiles it and some other custom drivers we have… =D

yes, linux-3.10.6-rt3 + 325.15 (nvidia-rt) booting ok, no lockups yet, but I havn’t tested anything RT yet, so i don’t know for shure, have some other drivers I have to make work with 3.xx before any real RT-testing. (if it locks up, does everything lock up, is it random or any special events?)

Thanks a lot for the help!

My current setup is:
Intel(R) Core™2 Duo CPU E7400 @ 2.80GHz
4Gb DDR3 RAM
120Gb 330 series intel SSD
ASUS Mobo (don’t have the number avaliable)
ZOTAC GEFORCE GT 610 1GB DDR3 PCI-E DVI/HDMI SILENT

Running debian/squeeze 32 bit
$uname -a
Linux pwrxxx 3.10.6-rt3 #2 SMP PREEMPT RT Wed Aug 21 11:32:22 CEST 2013 i686 GNU/Linux

We have been running 2.6.33.7-rt29 up till now, but need new drivers for new hardware, so time to upgrade the kernel. =)
We run our own PLC-software for industrial automation on 500+ linux nodes with this setup, so stability is crucial, check it out if you’re into that sort of thing, it’s GNU GPLed; http://www.proview.se. =)

Hey ulflj,

1st, I’m glad it is working. I assume you’ve run your standard GLX type stuff; compositor, H/W accel video, etc? ← That is typically where (afaict) you’d experience a lockup, if the nvidia driver wasn’t patched. ~ that is my experience and the experience i have observed, as well. <although, my machines exhibit it in slightly different ways, not entirely predictable>

On Archlinux, the whole (building) proces for nvidia-rt seems quite a bit easier than debian. That is another reason i’m glad it worked out for you (since i am not the best person to ask) - it seems “quite involved” to get nvidia installed on a debian machine (well, if you are doing it yourself anyway). On Archlinux, the process is rather simple; nvidia-rt 325.15 - Pastebin.com … meet dependencies, grab sources, extract installer, cd into …/kernel, then patch, then run make (with proper parameters), then do the installation bits…done! (all with one command “makepkg -si” from within folder containing PKGBUILD. :)

Regarding your 500+ nodes setup, very interesting. I’ve talked to people in the past doing similar interesting things. ~ I’m curious, what differences so far have you noticed between 2.33.7-rt and 3.x-rt kernel?? (ie: have you bumped into any significant regressions and/or improvments?). This kind of thing fascinates me, although for myself - my entire interest/use of Linux-rt is for proaudio purposes; I have a rackmountPC that i use as a “sound module” (and/or DAW). I hack on Wine to better support my Commercial proaudio apps (VSTs mainly and especially this boxset; Bundles: KOMPLETE 14 STANDARD .

but I do use linux proaudio apps to (like Ardour, for example)… but even aside from proaudio, i don’t use generic linux kernels, I essentially boot into an rt kernel(for standard desktop stuff, work, etc), all of the time on any machine(s). ~ and while my stability may not be as crucial as yours - it still is certainly important (and actually, my RackmountPC does have to be 100% reliable, no glitches… my desktop isn’t as crucial though).

Thanks, haven’t run any graphical things actual, we don’t have very much that require any 3D/accel gfx-performance, we mainly runt nvidia because they have nice dual-screen feature and (earlier) very easy installation through their own installer (so much for that now ;).

As i, really haven’t tested our software on the 3.x kernel yet, i don’t know, as I said it’s mainly a convinient upgrade to get new drivers to support new hardware (network, sata controller etc).

I’ll try to post back when I’ve run som test, I’ve manage to patch up our main I/O-driver to work with the 3.xx-kernel now, so at least we can test som things.

Yes, that makes sense to me - Looking at Proview gave me the impression that OpenGL wasn’t in high use, nor would a compositing desktop or anything like that… However, you might still want to test that, as you never know; it (by ‘it’ a mean a lockup) could randomly crop up in the future …

Ah, i guess you are going to have some testing / work ahead of you ;) good luck!

Well, I don’t get the impression that any of the nvidia linux developers even read these threads - so i guess i will just have to wait a release or two to see if something is added or changes in the installer…

Archlinux packagers for nvidia-rt have added my 325xx-rt.patch to their packaging and everything appears to be okay for their users. I know more than a few people who are also having success using my rt-patch for nvidia…so at least things are working…

but yeah, it would still be nice to hear something from “someone in the know” on the recent changes to the driver / nv-linux.h

cheerz

Hey there,
searching for a suitable patch for 352 brought me to this 2-year-old thread.
Just for anyone who comes across this the same way:
Setting the aforementioned environment variable is sufficient now and the nvidia driver adjusts to the preempt kernel at compile time then.

#export IGNORE_PREEMPT_RT_PRESENCE=1
1 Like

Could you tell me, what system or what kernel you were using?
I was looking for any information regarding recent Nvidia drivers and the RT-Kernel but I found only your post that is not older than two years.
I tried with Ubuntu 14.04 and the 4.1.7-rt8 patch, but I always end up with a frozen system after the next restart when I install the Nvidia driver
I’d be very happy if somebody could give me some advise.

I was now able to get a working system as well. I used the newest beta driver 358 with the aforementioned environment variable set.
My problem was that the driver seems to not support more than one graphics cards with Linux RT at the moment.
A patching as described here is not needed anymore.