OpenWRT Gets Forked By Some Of Its Own Developers As LEDE Project

[Djhg2000]

Their make is a huge hack that pulls down source and compiles stuff to make toolchain and stuff.

It's actually cool if you don't get errors. If you get errors, its very funny (not) to get it working again as NONE will EVER give a fuck about your messages.

This isn't what you were looking for. (jedi move)
What you and most people are looking for isn't building from source, but using the imagebuilder system from trunk download page (i.e. it will build the image pulling down the trunk precompiled packages, if existing).

Much faster, but you must have a list of packages you want to integrate, which has to be gathered by hand or by looking at what is in your .config.

Dependencies work, so you don't need to specify half the system.

go here https://downloads.openwrt.org/snapshots/trunk/ select the correct architecture/device-family and look for a tar.gz package with "ImageBuilder" in the name.

Then read the wiki page on how to use it https://wiki.openwrt.org/doc/howto/o...mware.generate
Or just look at the makefile in it to see what is written into it.



This is "normal". The process of pulling down stuff to compile follows dependencies if run as a single process, may or may not (usually not) follow dependencies when run as multiple processes.

I already shouted at them since I have a powerful multicore that I'd like to also use for compilation, their answer was "naah, I'm too lazy to do so" (literally).
I shouted again, Torvalds-style.

Sorry for the late response, I fell ill later that day (which probably explains my unusual need to use such strong wording) and I've been kind of loopy until today.

While I thank you for thoroughly explaining the build process, enabling the RTL8152/RTL8153 driver unfortunately does seem to require a recompile of the kernel source. None of that matters now because as it turns out I need a newer kernel for the new hardware anyway (the Intel NIC became supported in 4.0).

Currently I'm running a compile of the latest trunk while trying to wrestle the kernel from trunk into the local 15.05 tree (the latest build failed on applying patches after I copied over target/linux/x86/, before that I tried to simply edit the Makefile and got a barebone config). If you have any advice on switching kernels I'd be very grateful.

[starshipeleven]

Sorry for the late response, I fell ill later that day (which probably explains my unusual need to use such strong wording) and I've been kind of loopy until today.

While I thank you for thoroughly explaining the build process, enabling the RTL8152/RTL8153 driver unfortunately does seem to require a recompile of the kernel source. None of that matters now because as it turns out I need a newer kernel for the new hardware anyway (the Intel NIC became supported in 4.0).

Currently I'm running a compile of the latest trunk while trying to wrestle the kernel from trunk into the local 15.05 tree (the latest build failed on applying patches after I copied over target/linux/x86/, before that I tried to simply edit the Makefile and got a barebone config). If you have any advice on switching kernels I'd be very grateful.

Uhm, my bat-sense is tingling. Can you please state what device are you making firmware for?
Intel NIC, x86... sounds A LOT like PC hardware.

You sure a x86 networking-oriented distro like say Alpine or ClearOS or a NAS-oriented one like OpenMediaVault isn't better suited for it? OpenWRT is good for embedded devices with tiny onboard flash and RAM (and VERY crappy bootloaders), for x86 hardware there are much better options, see here a list https://en.wikipedia.org/wiki/List_o..._distributions

Still, I suspect that you got something wrong somewhere, afaik all non-essential drivers are compiled as modules in OpenWRT (to keep the kernel size down to a minimum, as it only needs to initialize the board and look at the rootfs where it can find all the rest of the stuff).

Anyway, if you tell me what device is that, I'll see what I can do. No promises of course.

[Djhg2000]

Uhm, my bat-sense is tingling. Can you please state what device are you making firmware for?
Intel NIC, x86... sounds A LOT like PC hardware.

You sure a x86 networking-oriented distro like say Alpine or ClearOS or a NAS-oriented one like OpenMediaVault isn't better suited for it? OpenWRT is good for embedded devices with tiny onboard flash and RAM (and VERY crappy bootloaders), for x86 hardware there are much better options, see here a list https://en.wikipedia.org/wiki/List_o..._distributions

Still, I suspect that you got something wrong somewhere, afaik all non-essential drivers are compiled as modules in OpenWRT (to keep the kernel size down to a minimum, as it only needs to initialize the board and look at the rootfs where it can find all the rest of the stuff).

Anyway, if you tell me what device is that, I'll see what I can do. No promises of course.

Certainly. It's a 6th gen Intel NUC which is going to replace an existing Netgear router running OpenWRT. The upgrade is because I couldn't get enough processing power out of the Netgear for heavy VPN traffic and a test run with a virtual machine passed with flying colors. I'd like to stick with OpenWRT because it fits in with the other network nodes and I'm fairly familiar with configuring it (started out with a WRT54G back in the good old days).

This is my first time trying to compile OpenWRT without using the tree supplied kernel. I've simply never needed anything more before and it was quite a while ago since the prebuilt images weren't good enough. The compiling experience is quite the long way from what I remember as well, but bearing in mind that was for the original hardware there was much less need for a flexible build system.

Anyway, the NIC in question is an Intel I219V (PCI ID 8086:1570) and the driver was mainlined in 4.0 . It may be possiblle to backport the patchset to 3.18, but that's an adventure I have yet to try (though I'd rather avoid that). The trunk build went fine and boots but I forgot to make it build all modules so the system ended up being... limited. I would much prefer a stable branch of OpenWRT with a swapped kernel because of the official feeds; everything but the NIC works fine with the 15.05 branch.

[starshipeleven]

The trunk build went fine and boots but I forgot to make it build all modules so the system ended up being... limited. I would much prefer a stable branch of OpenWRT with a swapped kernel because of the official feeds; everything but the NIC works fine with the 15.05 branch.

Trunk has more packages actually, and you can install them from repos, no need to build them (only need to build is if you want to integrate them in a system image, but you probably have ample storage space in that NIC).

Still, I think you can just change the repositories in the opkg config file to get packages from 15.05 instead.
I'm pretty sure that drivers and hardware-related stuff will NOT work as they are built for the specific kernel version, the same as with trunk, as they update their kernel the new driver packages in trunk don't work anymore until you update the kernel.
So you should probably leave the "core" feed with trunk and the others you can switch to 15.05 (never tried though so beware).

will look at this more this evening, when I can try with the imagebuilder for x86.

[Djhg2000]

Trunk has more packages actually, and you can install them from repos, no need to build them (only need to build is if you want to integrate them in a system image, but you probably have ample storage space in that NIC).

Still, I think you can just change the repositories in the opkg config file to get packages from 15.05 instead.
I'm pretty sure that drivers and hardware-related stuff will NOT work as they are built for the specific kernel version, the same as with trunk, as they update their kernel the new driver packages in trunk don't work anymore until you update the kernel.
So you should probably leave the "core" feed with trunk and the others you can switch to 15.05 (never tried though so beware).

will look at this more this evening, when I can try with the imagebuilder for x86.

I'm already building all of the kmod-* packages manually so there shouldn't be any unresolvable dependencies on the kernel, but I don't want to assume the core packages are backwards compatible with say, for instance, the luci-* packages. Because of the network issues I didn't have time to run it with an internet connection, but I'd much rather have at least the luci, openvpn and luci-app-openvpn packages (along with the XHCI stuff for supporting the RTL8153 adapter) integrated into the squashfs image. I'm currently running a new build of trunk which will hopefully include the desired packages.

By the way, is there any way to include config files in the squashfs image? It's not required for this system but it might be useful to know.

[starshipeleven]

I'm half sure that it should not need a recompile to work, anyway can you try the imagebuilder from here https://downloads.openwrt.org/snapshots/trunk/x86/64/

with the packages you have selected in your current .config?

I use a handy bash spell that reads the .config and gives you a nearly good list of packages to use with imagebuilder (you still need to remove something manually, but it's only a little).

#!/bin/bash
filename="/path/to/your/.config"
packages_for_make=$( sed -e 's/#.*$//' -e '/^$/d' -e 's/#.*$//' "$filename" | grep CONFIG_PACKAGE | sed -e 's/\CONFIG_PACKAGE_//g' -e 's/\=y//g' | tr '\n' ' ' )
echo $packages_for_make > "$filename"_filtered

Open the file and remove the rest of the crud.
Then after you filtered manually the few non-package names left, you can ask the imagebuilder to build the image with your package list, go in the folder where you extracted it and write:

make image PACKAGES="$( cat /path/to/the/text/file)"

This should be much faster as it just downloads tiny pre-built packages and integrates them.

If it cannot download a package it will throw an error.

By the way, is there any way to include config files in the squashfs image? It's not required for this system but it might be useful to know.

Yep. "In the build-systems topdir create a files/ directory and put the files there (e.g. files/etc/config/testconfig). Then rebuild the image."

https://forum.openwrt.org/viewtopic.php?id=22517

Placing config files in squashfs isn't terribly useful for your own devices imho, it's not like configs are going to be bigger than a few kB anyway.
That's probably useful for "default config" in commercial products or whatever, that after a full reset (and erase of erasable storage) still allows the device to have a default config to actually work at all.

[Djhg2000]

I'm half sure that it should not need a recompile to work, anyway can you try the imagebuilder from here https://downloads.openwrt.org/snapshots/trunk/x86/64/

with the packages you have selected in your current .config?

I use a handy bash spell that reads the .config and gives you a nearly good list of packages to use with imagebuilder (you still need to remove something manually, but it's only a little).

#!/bin/bash
filename="/path/to/your/.config"
packages_for_make=$( sed -e 's/#.*$//' -e '/^$/d' -e 's/#.*$//' "$filename" | grep CONFIG_PACKAGE | sed -e 's/\CONFIG_PACKAGE_//g' -e 's/\=y//g' | tr '\n' ' ' )
echo $packages_for_make > "$filename"_filtered

Open the file and remove the rest of the crud.
Then after you filtered manually the few non-package names left, you can ask the imagebuilder to build the image with your package list, go in the folder where you extracted it and write:

make image PACKAGES="$( cat /path/to/the/text/file)"

This should be much faster as it just downloads tiny pre-built packages and integrates them.

If it cannot download a package it will throw an error.

I found the trunk version wouldn't build anymore due to one of the repositories being down, so I used the previous image (with all the kmod-* packages) in a virtual machine to get an image up and running with the rest of the packages. Once that was done the new machine became a drop-in replacement of the existing router. The script I ran to generate the image was very similar, it grepped all the package files beginning with kmod- from the output of find, then stripped away the path and extension before saving it to a temporary file which I could feed to imagebuilder.

Yep. "In the build-systems topdir create a files/ directory and put the files there (e.g. files/etc/config/testconfig). Then rebuild the image."

https://forum.openwrt.org/viewtopic.php?id=22517

Placing config files in squashfs isn't terribly useful for your own devices imho, it's not like configs are going to be bigger than a few kB anyway.
That's probably useful for "default config" in commercial products or whatever, that after a full reset (and erase of erasable storage) still allows the device to have a default config to actually work at all.

Thanks, that's very useful to know. As you said the application would be for having a persistent default config, but it's not worth the trouble in this case. I'm getting more than just a little sidetracked here but is it also possible to have the squashfs and jffs2 images on different storage devices? For instance the squashfs image on an internal ROM and the jffs2 image on an SD card? If so how would OpenWRT go about locating the jffs2 partition?

[starshipeleven]

I'm getting more than just a little sidetracked here but is it also possible to have the squashfs and jffs2 images on different storage devices? For instance the squashfs image on an internal ROM and the jffs2 image on an SD card? If so how would OpenWRT go about locating the jffs2 partition?

That's a very weird need and I don't have a canned answer, you would have to muck around with the initialization config files changing the /dev/something/something of something or somesuch.
this is the boot process (so you know what file you need to look into) https://wiki.openwrt.org/doc/techref/process.boot
These are the config files involved https://wiki.openwrt.org/doc/howto/notuci.config

The altered files should be integrated in the same way as above. (never tried but it should work)

What is easy and proven (and provides a full recovery enviroment in case the second memory device or the filesystem or the firmware in it has issues) is just doing an extroot https://wiki.openwrt.org/doc/howto/extroot
That's what is routinely done on cheap routers with a USB port. Basic firmware that extroots inside a multiGB usb drive where it can install all the packages in the repos with space to spare.

[Djhg2000]

That's a very weird need and I don't have a canned answer, you would have to muck around with the initialization config files changing the /dev/something/something of something or somesuch.
this is the boot process (so you know what file you need to look into) https://wiki.openwrt.org/doc/techref/process.boot
These are the config files involved https://wiki.openwrt.org/doc/howto/notuci.config

The altered files should be integrated in the same way as above. (never tried but it should work)

What is easy and proven (and provides a full recovery enviroment in case the second memory device or the filesystem or the firmware in it has issues) is just doing an extroot https://wiki.openwrt.org/doc/howto/extroot
That's what is routinely done on cheap routers with a USB port. Basic firmware that extroots inside a multiGB usb drive where it can install all the packages in the repos with space to spare.

It's just that I just randomly thought of devices like the WRT54Gv5 where you only have 2MB internal flash and the ability to hack in an SD card on the internal SPI bus. It's not a very relevant device anymore but similar situations may arise in the future. Like if you had a handheld device with very limited storage you wanted to run an X server on.

With smartphones dominating the phone market and devices like the DragonBox Pyra and the GPD Win about to start shipping (and the first of which I've preordered) I don't think I will ever even need to consider building these setups. But if I don't investigate these things now there's guaranteed to arise a situation in the future where not knowing this will come back to haunt me.

Other than the build system, which I have very mixed feelings about, OpenWRT is still an awesome distro for embedded devices in my opinion. On the original topic of the thread, let's see what LEDE can bring to the table

[starshipeleven]

It's just that I just randomly thought of devices like the WRT54Gv5 where you only have 2MB internal flash and the ability to hack in an SD card on the internal SPI bus. It's not a very relevant device anymore but similar situations may arise in the future. Like if you had a handheld device with very limited storage you wanted to run an X server on.

With smartphones dominating the phone market and devices like the DragonBox Pyra and the GPD Win about to start shipping (and the first of which I've preordered) I don't think I will ever even need to consider building these setups. But if I don't investigate these things now there's guaranteed to arise a situation in the future where not knowing this will come back to haunt me.

Consider that there are gigabit eth tri-external-antenna wifi-n routers with usb ports supported by openwrt that sell for 20$.
Unless it's paramount for the human survival on the planet, I'm more inclined to replace the obsolete crap with that instead of jumping through hoops for weeks to get them running like garbage.

Other than the build system, which I have very mixed feelings about, OpenWRT is still an awesome distro for embedded devices in my opinion. On the original topic of the thread, let's see what LEDE can bring to the table

Yeah, I had various ideas to make OpenWRT better but was always pissed by their closed and confusing nature (I'm not happy to work on things that then aren't getting used), hoping LEDE fixes that.