Announcement

Collapse
No announcement yet.

Need Some Crossfire Help

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Need Some Crossfire Help

    Hello,

    I'm sure I'm like 1000th person to come on here and ask but just not sure what I'm doing wrong here.

    Linux 3.2.1-gentoo-r2 x86_64 AMD FX(tm)-8150 Eight-Core Processor

    ~ $ aticonfig --lsa

    * 0. 03:00.0 ATI Radeon HD 5900 Series
    1. 04:00.0 ATI Radeon HD 5900 Series
    2. 0a:00.0 ATI Radeon HD 5800 Series

    ~ $ aticonfig --lsch

    CrossFire chain for adapter 0, status: enabled
    0. 03:00.0 ATI Radeon HD 5900 Series
    2. 0a:00.0 ATI Radeon HD 5800 Series
    1. 04:00.0 ATI Radeon HD 5900 Series

    CrossFire chain for adapter 2, status: enabled
    2. 0a:00.0 ATI Radeon HD 5800 Series
    0. 03:00.0 ATI Radeon HD 5900 Series
    1. 04:00.0 ATI Radeon HD 5900 Series

    ~ $ aticonfig --lscc

    Master adapter: 0. 03:00.0 ATI Radeon HD 5900 Series
    Candidates: 1. 04:00.0 ATI Radeon HD 5900 Series

    Master adapter: 1. 04:00.0 ATI Radeon HD 5900 Series
    Candidates: 1. 04:00.0 ATI Radeon HD 5900 Series

    Master adapter: 2. 0a:00.0 ATI Radeon HD 5800 Series
    Candidates: 1. 04:00.0 ATI Radeon HD 5900 Series

    Okay so here is where I don't know what to do next lol

    ~ $ aticonfig --lscs

    Candidate Combination:
    Master: 3:0:0
    Slave: 4:0:0
    CrossFire is disabled on current device
    CrossFire Diagnostics:
    CrossFire can work with P2P mapping through GART
    Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
    Candidate Combination:
    Master: 3:0:0
    Slave: 4:0:0
    CrossFire is disabled on current device
    CrossFire Diagnostics:
    CrossFire can work with P2P mapping through GART
    Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
    Candidate Combination:
    Master: 3:0:0
    Slave: 4:0:0
    CrossFire is disabled on current device
    CrossFire Diagnostics:
    CrossFire can work with P2P mapping through GART
    Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA

    Yes I have rebooted multiple times now any help would be greatly appreciated I'll give the output to lspci below.
    Keep in Mind I'm running it on a Asus Crosshair V Formula 990FX chipset (But even in Winblows it picksups as an 890 so not too concerned)


    # lspci

    00:00.0 Host bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
    00:00.2 IOMMU: Advanced Micro Devices [AMD] nee ATI RD990 I/O Memory Management Unit (IOMMU)
    00:02.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port B)
    00:05.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port E)
    00:06.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port F)
    00:07.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port G)
    00:09.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port H)
    00:0a.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx1 port A)
    00:0b.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (NB-SB link)
    00:11.0 SATA controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
    00:12.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
    00:12.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
    00:13.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
    00:13.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
    00:14.0 SMBus: Advanced Micro Devices [AMD] nee ATI SBx00 SMBus Controller (rev 42)
    00:14.3 ISA bridge: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
    00:14.4 PCI bridge: Advanced Micro Devices [AMD] nee ATI SBx00 PCI to PCI Bridge (rev 40)
    00:14.5 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
    00:16.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
    00:16.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
    00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
    00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
    00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
    00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
    00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
    00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
    01:00.0 PCI bridge: PLX Technology, Inc. PEX 8647 48-Lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev bb)
    02:04.0 PCI bridge: PLX Technology, Inc. PEX 8647 48-Lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev bb)
    02:08.0 PCI bridge: PLX Technology, Inc. PEX 8647 48-Lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev bb)
    03:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Hemlock [Radeon HD 5900 Series]
    03:00.1 Audio device: Advanced Micro Devices [AMD] nee ATI Cypress HDMI Audio [Radeon HD 5800 Series]
    04:00.0 Display controller: Advanced Micro Devices [AMD] nee ATI Hemlock [Radeon HD 5900 Series]
    05:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
    06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
    07:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
    08:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
    09:00.0 Audio device: Creative Labs X-Fi Titanium series [EMU20k2] (rev 04)
    0a:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Cypress XT [Radeon HD 5870]
    0a:00.1 Audio device: Advanced Micro Devices [AMD] nee ATI Cypress HDMI Audio [Radeon HD 5800 Series]
    4
    Very Helpful
    50.00%
    2
    Neutral
    50.00%
    2
    Confusing
    0.00%
    0
    Intriguing
    0.00%
    0

  • #2
    First Q - if I remember our product numbers correctly that's 5 GPUs worth of boards there -- I thought we only supported up to 4.

    I haven't played with crossfire at all but first thing I would try is reducing the number of GPUs.

    Comment


    • #3
      Nope

      Originally posted by bridgman View Post
      First Q - if I remember our product numbers correctly that's 5 GPUs worth of boards there -- I thought we only supported up to 4.

      I haven't played with crossfire at all but first thing I would try is reducing the number of GPUs.
      lol nope that be interesting though, 5970 has 2 GPU's and the 5870 has 1 so it's Tri-Fire even in windblows only 4 gpus is supported I believe , this motherboard does support Quad cards not sure how the drivers would react too putting in 4 dual gpu cards even if you could get it work would most likely be a bottle neck anyway so wouldn't be worth it not to mention you can substitute your tower for a small heater at the point lol
      Last edited by Acidphase; 01-26-2012, 09:44 AM.

      Comment


      • #4
        Whoops, one of those 5970 devices was an audio controller. Reply in haste, repent in leisure

        My guess is that 2-way gets more testing than 3-way, might be worth trying with just the 5970 first. There are some slightly different aticonfig --initial parms required, something like adapters=all IIRC, check that as well.

        Comment


        • #5
          lol

          Originally posted by bridgman View Post
          Whoops, one of those 5970 devices was an audio controller. Reply in haste, repent in leisure

          My guess is that 2-way gets more testing than 3-way, might be worth trying with just the 5970 first. There are some slightly different aticonfig --initial parms required, something like adapters=all IIRC, check that as well.
          Yep already did all that xorg.conf all setup (fyi aticonfig won't even allow you to set crossfire chains up unless the devices are already specified in the xorg.conf) and tired with just the 5970 same outcome.

          ~ $ aticonfig --lsch

          CrossFire chain for adapter 0, status: enabled
          0. 03:00.0 ATI Radeon HD 5900 Series
          1. 04:00.0 ATI Radeon HD 5900 Series

          ~ $ aticonfig --lscs
          Candidate Combination:
          Master: 3:0:0
          Slave: 4:0:0
          CrossFire is disabled on current device
          CrossFire Diagnostics:
          CrossFire can work with P2P mapping through GART
          Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
          Candidate Combination:
          Master: 3:0:0
          Slave: 4:0:0
          CrossFire is disabled on current device
          CrossFire Diagnostics:
          CrossFire can work with P2P mapping through GART
          Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
          Candidate Combination:
          Master: 3:0:0
          Slave: 4:0:0
          CrossFire is disabled on current device
          CrossFire Diagnostics:
          CrossFire can work with P2P mapping through GART
          Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
          Last edited by Acidphase; 01-26-2012, 11:05 AM.

          Comment


          • #6
            Nothing huh lol

            I didn't have time to mess with it today I will continue if I have time tomorrow.
            If I get things going I'm going end up updating the Gentoo wiki on FGLRX because it's piss poor on crossfire setups.

            Comment


            • #7
              Bridgman, AMD Dual Graphics is supported by Catalyst for Linux?

              Acer 7560G, AMD A8 3500M APU (Radeon HD 6620G) and Radeon HD 6650M.
              Detected GPU's:
              00:01.0 VGA compatible controller: ATI Technologies Inc Device 9641
              01:00.0 VGA compatible controller: ATI Technologies Inc NI Whistler [AMD Radeon HD 6600M Series]

              But 6650M is not detected like candidate for CrossFire chain:
              amdconfig --list-crossfire-candidates
              Master adapter: 0. 00:01.0 AMD Radeon HD 6620G
              Candidates: none

              I try to create CrossFire chain anyway:
              amdconfig --add-crossfire-chain --adapter=all
              amdconfig --crossfire=on --adapter=all

              CrossFire chain is enabled:
              amdconfig --lsch
              CrossFire chain for adapter 0, status: enabled
              0. 00:01.0 AMD Radeon HD 6620G
              1. 01:00.0 AMD Radeon 6600M and 6700M Series

              But CrossFire expectedly doesn't work:

              amdconfig --lscs
              Candidate Combination:
              Master: 0:0:0
              Slave: 0:0:0
              CrossFire is disabled on current device
              CrossFire Diagnostics:
              CrossFire can work with P2P mapping through GART

              Comment


              • #8
                In your situation I'd say no.

                Originally posted by RussianNeuroMancer View Post
                Bridgman, AMD Dual Graphics is supported by Catalyst for Linux?

                Acer 7560G, AMD A8 3500M APU (Radeon HD 6620G) and Radeon HD 6650M.
                Detected GPU's:
                00:01.0 VGA compatible controller: ATI Technologies Inc Device 9641
                01:00.0 VGA compatible controller: ATI Technologies Inc NI Whistler [AMD Radeon HD 6600M Series]

                But 6650M is not detected like candidate for CrossFire chain:
                amdconfig --list-crossfire-candidates
                Master adapter: 0. 00:01.0 AMD Radeon HD 6620G
                Candidates: none

                I try to create CrossFire chain anyway:
                amdconfig --add-crossfire-chain --adapter=all
                amdconfig --crossfire=on --adapter=all

                CrossFire chain is enabled:
                amdconfig --lsch
                CrossFire chain for adapter 0, status: enabled
                0. 00:01.0 AMD Radeon HD 6620G
                1. 01:00.0 AMD Radeon 6600M and 6700M Series

                But CrossFire expectedly doesn't work:

                amdconfig --lscs
                Candidate Combination:
                Master: 0:0:0
                Slave: 0:0:0
                CrossFire is disabled on current device
                CrossFire Diagnostics:
                CrossFire can work with P2P mapping through GART
                Even under normal a situation with two discreet cards they have to be in the same Series to X-fire with exception of the a couple of 5700 and 6700 series cards. Being one of your
                GPU's is really an APU not sure how that would work out. From my understanding it possible to X-fire with a APU but only with certain combinations haven't see a chart for those
                situations though and at that was in Win7 x64.
                Even if you do get one up and going there has been claims of Micro Stuttering. However it is said you can add a second discreet card and that wouldn't be a problem.

                Comment


                • #9
                  Maybe?

                  So maybe in the near future more than 2 GPU crossfire setups may be supported in Linux a little better for now seems like pipe dreams but with news from the below link gives you hope.

                  AMD OpenCL(TM) APP SDK 2.6 and AMD Catalyst(TM) Developer Driver Available for Download

                  Comment


                  • #10
                    Originally posted by Acidphase View Post
                    Being one of your GPU's is really an APU not sure how that would work out.
                    There is even special marketing name for "APU + discrete GPU" CrossFireX - AMD Dual Graphics.
                    Originally posted by Acidphase View Post
                    From my understanding it possible to X-fire with a APU but only with certain combinations
                    Yes, and combination in this laptop is supported and even listed on AMD official web-site.

                    Comment


                    • #11
                      Originally posted by RussianNeuroMancer View Post
                      There is even special marketing name for "APU + discrete GPU" CrossFireX - AMD Dual Graphics.
                      Yes, and combination in this laptop is supported and even listed on AMD official web-site.
                      In that case:

                      Code:
                      # aticonfig --adapter=all --cfd
                      
                      # mv /etc/X11/xorg.conf /etc/X11/xorg.conf.old
                      
                      # aticonfig --adapter=0,1 --initial 
                      
                      # reboot (in ubuntu shutdown -r now assuming your in tty*)
                      Once back in DE or if you're like me and don't have xdm added to the runlevel default go head and start X

                      From terminal or menu load amdcccle as root/super user If you you're not using more than on screen in may prompt you that it has to make changes let it do so. Also while you're there see if you have a Crossfire expandable menu
                      I've been told by a few people that you can enable it straight from there is so great do so then reboot.

                      The automatic changes basically removes "right of 0 0" in the server layout section which you need because because either you're going to run in a Big Desktop mode or and mutli desktop mode.
                      That's why you're seeing the 0:0:0 which isn't the correct BUS ID for your adapters. instead of:

                      Code:
                      Candidate Combination:
                      Master: 0:0:0
                      Slave: 0:0:0
                      CrossFire is disabled on current device
                      CrossFire Diagnostics:
                      CrossFire can work with P2P mapping through GART
                      You should be seeing something like:
                      Code:
                      Candidate Combination:
                      Master: 0:1:0
                      Slave: 1:0:0
                      CrossFire is disabled on current device
                      CrossFire Diagnostics:
                      CrossFire can work with P2P mapping through GART
                      Now if you're seeing enabled via:
                      Code:
                      amdconfig --lsch
                      CrossFire chain for adapter 0, status: enabled
                      0. 00:01.0 AMD Radeon HD 6620G
                      1. 01:00.0 AMD Radeon 6600M and 6700M Series
                      When your amdconfig --lscs is showing:
                      Code:
                      Candidate Combination:
                      Master: 0:1:0
                      Slave: 1:0:0
                      CrossFire is disabled on current device
                      CrossFire Diagnostics:
                      CrossFire can work with P2P mapping through GART
                      Meaning still "disabled" rather than enabled your in the same boat as me lol. I even tried too crossfire just the 5970 which is a X2 Adapter and should work but still looks as stated above.
                      Like:
                      Code:
                      ~ $ aticonfig --lsch
                      
                      CrossFire chain for adapter 0, status: enabled
                        0. 03:00.0 ATI Radeon HD 5900 Series
                        1. 04:00.0 ATI Radeon HD 5900 Series
                      
                       ~ $ aticonfig --lscs
                          Candidate Combination: 
                          Master: 3:0:0 
                          Slave: 4:0:0 
                          CrossFire is disabled on current device
                          CrossFire Diagnostics:
                          CrossFire can work with P2P mapping through GART
                          Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA 
                          Candidate Combination: 
                          Master: 3:0:0 
                          Slave: 4:0:0 
                          CrossFire is disabled on current device
                          CrossFire Diagnostics:
                          CrossFire can work with P2P mapping through GART
                          Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA 
                          Candidate Combination: 
                          Master: 3:0:0 
                          Slave: 4:0:0 
                          CrossFire is disabled on current device
                          CrossFire Diagnostics:
                          CrossFire can work with P2P mapping through GART
                          Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
                      Now I've been told by someone if you have a IOMMU device it needs to be disabled in BIOS and in the kernel command of your boot loader something like:
                      Code:
                      kernel /vmlinuz real_root=/dev/sda3 iommu=off
                      There is also intel_iommu and amd_iommu which are the devices I'm assuming that your disabling in bios as it seems I don't have an option to set iommu=off cause everytime I do I lose control over my input devices. However there are other known options than just off that I haven't had time to use and investigate exactly what each one does. I may be able to only disable the portion of the IOMMU that's keeping the mapping to be made via Gart pr P2P to allow a Crossfire configuration.

                      If someone on the forums here, possibly a Developer or someone very knowledgeable in IOMMU devices and settings can explain and save me, you, and future users allot of time it would be appreciated =P

                      Here of some of the known options I know of but I'm not sure if their all related in the sense that AMD64 IOMMU embedded options are the same for amd_iommu and visa versa as I said when I get time I will research it further.

                      AMD64 Embedded IOMMU:
                      IOMMU (input/output memory management unit)

                      Currently four x86-64 PCI-DMA mapping implementations exist:

                      1. <arch/x86_64/kernel/pci-nommu.c>: use no hardware/software IOMMU at all
                      (e.g. because you have < 3 GB memory).
                      Kernel boot message: "PCI-DMA: Disabling IOMMU"

                      2. <arch/x86/kernel/amd_gart_64.c>: AMD GART based hardware IOMMU.
                      Kernel boot message: "PCI-DMA: using GART IOMMU"

                      3. <arch/x86_64/kernel/pci-swiotlb.c> : Software IOMMU implementation. Used
                      e.g. if there is no hardware IOMMU in the system and it is need because
                      you have >3GB memory or told the kernel to us it (iommu=soft))
                      Kernel boot message: "PCI-DMA: Using software bounce buffering
                      for IO (SWIOTLB)"

                      4. <arch/x86_64/pci-calgary.c> : IBM Calgary hardware IOMMU. Used in IBM
                      pSeries and xSeries servers. This hardware IOMMU supports DMA address
                      mapping with memory protection, etc.
                      Kernel boot message: "PCI-DMA: Using Calgary IOMMU"

                      iommu=[<size>][,noagp][,off][,force][,noforce][,leak[=<nr_of_leak_pages>]
                      [,memaper[=<order>]][,merge][,forcesac][,fullflush][,nomerge]
                      [,noaperture][,calgary]

                      General iommu options:
                      off Don't initialize and use any kind of IOMMU.
                      noforce Don't force hardware IOMMU usage when it is not needed.
                      (default).
                      force Force the use of the hardware IOMMU even when it is
                      not actually needed (e.g. because < 3 GB memory).
                      soft Use software bounce buffering (SWIOTLB) (default for
                      Intel machines). This can be used to prevent the usage
                      of an available hardware IOMMU.

                      iommu options only relevant to the AMD GART hardware IOMMU:
                      <size> Set the size of the remapping area in bytes.
                      allowed Overwrite iommu off workarounds for specific chipsets.
                      fullflush Flush IOMMU on each allocation (default).
                      nofullflush Don't use IOMMU fullflush.
                      leak Turn on simple iommu leak tracing (only when
                      CONFIG_IOMMU_LEAK is on). Default number of leak pages
                      is 20.
                      memaper[=<order>] Allocate an own aperture over RAM with size 32MB<<order.
                      (default: order=1, i.e. 64MB)
                      merge Do scatter-gather (SG) merging. Implies "force"
                      (experimental).
                      nomerge Don't do scatter-gather (SG) merging.
                      noaperture Ask the IOMMU not to touch the aperture for AGP.
                      forcesac Force single-address cycle (SAC) mode for masks <40bits
                      (experimental).
                      noagp Don't initialize the AGP driver and use full aperture.
                      allowdac Allow double-address cycle (DAC) mode, i.e. DMA >4GB.
                      DAC is used with 32-bit PCI to push a 64-bit address in
                      two cycles. When off all DMA over >4GB is forced through
                      an IOMMU or software bounce buffering.
                      nodac Forbid DAC mode, i.e. DMA >4GB.
                      panic Always panic when IOMMU overflows.
                      calgary Use the Calgary IOMMU if it is available

                      iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
                      implementation:
                      swiotlb=<pages>[,force]
                      <pages> Prereserve that many 128K pages for the software IO
                      bounce buffering.
                      force Force all IO through the software TLB.

                      Settings for the IBM Calgary hardware IOMMU currently found in IBM
                      pSeries and xSeries machines:

                      calgary=[64k,128k,256k,512k,1M,2M,4M,8M]
                      calgary=[translate_empty_slots]
                      calgary=[disable=<PCI bus number>]
                      panic Always panic when IOMMU overflows

                      64k,...,8M - Set the size of each PCI slot's translation table
                      when using the Calgary IOMMU. This is the size of the translation
                      table itself in main memory. The smallest table, 64k, covers an IO
                      space of 32MB; the largest, 8MB table, can cover an IO space of
                      4GB. Normally the kernel will make the right choice by itself.

                      translate_empty_slots - Enable translation even on slots that have
                      no devices attached to them, in case a device will be hotplugged
                      in the future.

                      disable=<PCI bus number> - Disable translation on a given PHB. For
                      example, the built-in graphics adapter resides on the first bridge
                      (PCI bus number 0); if translation (isolation) is enabled on this
                      bridge, X servers that access the hardware directly from user
                      space might stop working. Use this option if you have devices that
                      are accessed from userspace directly on some PCI host bridge.
                      IA64 Embedded IOMMU:
                      inttest= [IA64]

                      iommu=option[,option..] [X86-64]
                      off Disable IOMMU.
                      force Unconditionally enable IOMMU.
                      noforce Disable IOMMU and IOMMU merging, by default.
                      biomerge Unconditionally enable IOMMU, IOMMU merging,
                      and set BIO IOMMU vmerge boundary to 4096.
                      panic Panic on IOMMU overflow.
                      nopanic Do not panic on IOMMU overflow.
                      merge Unconditionally enable IOMMU, IOMMU merging.
                      nomerge Disable IOMMU merging.
                      forcesac Force single address cycle (SAC, 32-bit).
                      allowdac Permit dual address cycle (DAC, 64-bit).
                      nodac Forbid dual address cycle (DAC, 64-bit).
                      soft Enable swiotlb.
                      calgary Use Calgary IOMMU.

                      (GART-only options follow...)
                      <NNN> Specify size of remapping area.
                      fullflush Disable optimizing flushing strategy.
                      nofullflush Enable optimizing flushing strategy.
                      noagp Use entire aperture, AGP isn't using it.
                      noaperture Disable aperture fixups / hole init.
                      memaper=<N> malloc an aperture of order N.
                      AMD_IOMMU Device:
                      amd_iommu= [HW,X86-84]
                      Pass parameters to the AMD IOMMU driver in the system.
                      Possible values are:
                      fullflush - enable flushing of IO/TLB entries when
                      they are unmapped. Otherwise they are
                      flushed before they will be reused, which
                      is a lot of faster
                      off - do not initialize any AMD IOMMU found in
                      the system
                      INTEL_IOMMU Device:
                      intel_iommu= [DMAR] Intel IOMMU driver (DMAR) option
                      on
                      Enable intel iommu driver.
                      off
                      Disable intel iommu driver.
                      igfx_off [Default Off]
                      By default, gfx is mapped as normal device. If a gfx
                      device has a dedicated DMAR unit, the DMAR unit is
                      bypassed by not enabling DMAR with this option. In
                      this case, gfx device will use physical address for
                      DMA.
                      forcedac [x86_64]
                      With this option iommu will not optimize to look
                      for io virtual address below 32 bit forcing dual
                      address cycle on pci bus for cards supporting greater
                      than 32 bit addressing. The default is to look
                      for translation below 32 bit and if not available
                      then look in the higher range.
                      strict [Default Off]
                      With this option on every unmap_single operation will
                      result in a hardware IOTLB flush operation as opposed
                      to batching them for performance.
                      sp_off [Default Off]
                      By default, super page will be supported if Intel IOMMU
                      has the capability. With this option, super page will
                      not be supported.
                      intremap= [X86-64, Intel-IOMMU]
                      Format: { on (default) | off | nosid }
                      on enable Interrupt Remapping (default)
                      off disable Interrupt Remapping
                      nosid disable Source ID checking

                      inttest= [IA64]

                      iomem= Disable strict checking of access to MMIO memory
                      strict regions from userspace.
                      relaxed

                      iommu= [x86]
                      off
                      force
                      noforce
                      biomerge
                      panic
                      nopanic
                      merge
                      nomerge
                      forcesac
                      soft
                      pt [x86, IA64]
                      Last edited by Acidphase; 02-11-2012, 06:21 PM.

                      Comment


                      • #12
                        Originally posted by RussianNeuroMancer View Post
                        There is even special marketing name for "APU + discrete GPU" CrossFireX - AMD Dual Graphics.
                        Yes, and combination in this laptop is supported and even listed on AMD official web-site.
                        Yep, looks like a supported combination :

                        http://www.amd.com/us/products/techn...aphics.aspx#/2

                        What's not so clear is the degree of support under Linux... footnote 1 suggests that it might be supported but a restart of X is required to enable/disable :

                        http://www.amd.com/us/products/techn...aphics.aspx#/5

                        It's not obvious to me what the right sequence of amdconfig commands and restarts is though...

                        Guess the first thing I would try is :

                        - dump out amdconfig options to see if there are any --initial parms specific to crossfire/dual graphics

                        - stop X
                        - run amdconfig --initial adapters=all from console, along with any specific-to-crossfire stuff I found in the amdconfig output
                        - check to see if crossfire candidate is found
                        - if so enable crossfire
                        - start X

                        I'll ask around and see if there is a better guide, or maybe something in the release/install notes.
                        Last edited by bridgman; 02-11-2012, 07:51 PM.

                        Comment


                        • #13
                          Originally posted by bridgman View Post
                          Yep, looks like a supported combination :

                          http://www.amd.com/us/products/techn...aphics.aspx#/2

                          What's not so clear is the degree of support under Linux... footnote 1 suggests that it might be supported but a restart of X is required to enable/disable :

                          http://www.amd.com/us/products/techn...aphics.aspx#/5

                          It's not obvious to me what the right sequence of amdconfig commands and restarts is though...

                          Guess the first thing I would try is :

                          - dump out amdconfig options to see if there are any --initial parms specific to crossfire/dual graphics

                          - stop X
                          - run amdconfig --initial adapters=all from console, along with any specific-to-crossfire stuff I found in the amdconfig output
                          - check to see if crossfire candidate is found
                          - if so enable crossfire
                          - start X

                          I'll ask around and see if there is a better guide, or maybe something in the release/install notes.
                          From the guides I have seen relating to Crossfire and X2 cards that's when it said to use the --adapter=all however in the same guides in Bold said "Do not use with more than one display adapter"

                          So being in his situation I'd figure --adapter=0,1 would be the correct way but --adapter=all would have the same outcome so makes no sense really either way.

                          I have been through these commands with a fine toothed comb like I stated previously someone needs to explain the whole Gart and P2P bit a little better which is related to the IOMMU and AGP/PCI DMA.
                          As far the sequence the one I told him and how to do it should be fine as it will add the chain or should add it after the restart of X and enable it as well as it did with my 5970 card.
                          Our problem isn't enabling it or adding the chain that part is easy the problem is getting it to actually initiate after it's enabled. Which again goes back to the Gard and P2P which you can see with the
                          aticonfig --lscs command.

                          As far the --initial and crossfire commands their pretty straight forward.
                          Code:
                            5. Multiple display adapters.
                                 List adapters :  aticonfig --list-adapters
                                 Init 0 and 2  :  aticonfig --adapter=0,2 --initial
                                 Init all      :  aticonfig --adapter=all --initial
                          Crossfire Specific:
                          Code:
                          Multiple display adapter options:
                            Following options are used for querying and setting up multiple
                            display adapters that are installed for multihead or Crossfire
                            configurations.
                            --lsa, --list-adapters
                                  Lists all detected and supported display adapters.
                                  The default adapter (used when --adapter is not specified)
                                  will be indicated with a "*" next to it.
                            --adapter=ADAPTERLIST
                                  Selects which adapters returned by --list-adapters should
                                  be affected by other aticonfig options.  ADAPTERLIST contains
                                  either a comma-seperated sequence of the index numbers of the
                                  adapters to be affected or else contains the keyword "all" to
                                  select all the adapters.  If --adapter is missing, only the
                                  default adapter will be affected.
                            --lscc, --list-crossfire-candidates
                                  Queries the driver to determine the pool of available devices that can
                                  can be chained together for CrossFire.
                            --lscs, --list-crossfire-status
                                  List current Crossfire status (enabled or disabled) along with diagnostics
                                  information indicates the status of your system
                            --lsch, --list-crossfire-chains
                                  Lists the CrossFire chains that are currently defined along with their
                                  enabled state
                            --cfa, --add-crossfire-chain
                                  Defines a new CrossFire chain.  --adapter should contain the adapter
                                  chain definition, with the master adapter being the first entry and
                                  the slave adapters being the subsequent entries in order of priority.
                            --cfd, --delete-crossfire-chain
                                  Delete and existing defined CrossFire chain.  --adapter should list the
                                  master adapters of the chains to be deleted.  --adapter=all will delete
                                  all chain definitions.
                            --cf, --crossfire={on|off}
                                  Enables/disables CrossFire support on the currently defined CrossFire
                                  chains.  --adapter should list the master adapters to be enabled or
                                  disabled.
                            --cfl, --crossfire-logo={on|off}
                                  Enables/disables the appearance of the CrossFire Logo when rendering
                                  in CrossFire mode
                          Last edited by Acidphase; 02-11-2012, 09:45 PM.

                          Comment


                          • #14
                            Conclusion

                            It seems when you have the physical crossfire bridge connected is where the Linux Kernel fails. In a TRI-Fire setup or more that is.

                            When the MSI-X sets up the IRQ for each device it's having a problem giving an IRQ for the 3rd GPU it isn't seeing it as a proper vga device.
                            However it' saying a bridge is possible.

                            Code:
                            dmesg | grep vga
                            [    0.233493] vgaarb: device added: PCI:0000:03:00.0,decodes=io+mem,owns=io+mem,locks=none
                            [    0.233493] vgaarb: device added: PCI:0000:0a:00.0,decodes=io+mem,owns=none,locks=none
                            [    0.233493] vgaarb: loaded
                            [    0.233493] vgaarb: bridge control possible 0000:0a:00.0
                            [    0.233493] vgaarb: bridge control possible 0000:03:00.0
                            [   49.834414] vgaarb: this pci device is not a vga device
                            [   49.834421] vgaarb: this pci device is not a vga device
                            [   49.834429] vgaarb: this pci device is not a vga device
                            [   49.834436] vgaarb: this pci device is not a vga device
                            [   49.834443] vgaarb: this pci device is not a vga device
                            [   49.834450] vgaarb: this pci device is not a vga device
                            [   49.834457] vgaarb: this pci device is not a vga device
                            [   49.834464] vgaarb: this pci device is not a vga device
                            [   49.834471] vgaarb: this pci device is not a vga device
                            [   49.834478] vgaarb: this pci device is not a vga device
                            [   49.834485] vgaarb: this pci device is not a vga device
                            [   49.834492] vgaarb: this pci device is not a vga device
                            [   49.834498] vgaarb: this pci device is not a vga device
                            [   49.834506] vgaarb: this pci device is not a vga device
                            [   49.834513] vgaarb: this pci device is not a vga device
                            [   49.834520] vgaarb: this pci device is not a vga device
                            [   49.834527] vgaarb: this pci device is not a vga device
                            [   49.834534] vgaarb: this pci device is not a vga device
                            [   49.834541] vgaarb: this pci device is not a vga device
                            [   49.834548] vgaarb: this pci device is not a vga device
                            [   49.834555] vgaarb: this pci device is not a vga device
                            [   49.834562] vgaarb: this pci device is not a vga device
                            [   49.834569] vgaarb: this pci device is not a vga device
                            [   49.834576] vgaarb: this pci device is not a vga device
                            [   49.834582] vgaarb: this pci device is not a vga device
                            [   49.834589] vgaarb: this pci device is not a vga device
                            [   49.834597] vgaarb: this pci device is not a vga device
                            [   49.834604] vgaarb: this pci device is not a vga device
                            [   49.834612] vgaarb: this pci device is not a vga device
                            [   49.834628] vgaarb: this pci device is not a vga device
                            [   49.834636] vgaarb: this pci device is not a vga device
                            [   49.834644] vgaarb: this pci device is not a vga device
                            [   49.834652] vgaarb: this pci device is not a vga device
                            [   49.834661] vgaarb: this pci device is not a vga device
                            [   49.834669] vgaarb: this pci device is not a vga device
                            [   49.834676] vgaarb: this pci device is not a vga device
                            [   49.834690] vgaarb: device changed decodes: PCI:0000:0a:00.0,olddecodes=io+mem,decodes=none:owns=none
                            [   49.834702] vgaarb: this pci device is not a vga device
                            Now a truncated output of
                            Code:
                            dmesg | grep pci
                            
                            [   50.438714] fglrx_pci 0000:03:00.0: irq 100 for MSI/MSI-X
                            [   50.454599] fglrx_pci 0000:04:00.0: irq 101 for MSI/MSI-X

                            Quite odd , even odder is the output of aticonfig --lscs especially compared against the above state that a bridge is possible.

                            Code:
                            aticonfig --lscs
                                Candidate Combination: 
                                Master: 3:0:0 
                                Slave: 4:0:0 
                                CrossFire is disabled on current device
                                CrossFire Diagnostics:
                                CrossFire can work with P2P mapping through GART
                                Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA 
                                Candidate Combination: 
                                Master: 3:0:0 
                                Slave: 4:0:0 
                                CrossFire is disabled on current device
                                CrossFire Diagnostics:
                                CrossFire can work with P2P mapping through GART
                                Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA 
                                Candidate Combination: 
                                Master: 3:0:0 
                                Slave: 4:0:0 
                                CrossFire is disabled on current device
                                CrossFire Diagnostics:
                                CrossFire can work with P2P mapping through GART
                                Dongle Capabilities: support PASSTHROUGH |INTERLINK_SW_AFR | INTERLINK_AUTO_AFR | INTERLINK_BLACKING | INTERLINK_SUPERAA
                            What's odd is the 3 entry as the PCI BUS ID's are wrong on a Duel GPU setup would just see the first two entries. Whic makes the next command even more odd lol.

                            Code:
                            aticonfig --lscc
                            
                            Master adapter:  0. 03:00.0 ATI Radeon HD 5900 Series
                                Candidates:  1. 04:00.0 ATI Radeon HD 5900 Series
                                           
                            Master adapter:  1. 04:00.0 ATI Radeon HD 5900 Series
                                Candidates:  1. 04:00.0 ATI Radeon HD 5900 Series
                                           
                            Master adapter:  2. 0a:00.0 ATI Radeon HD 5800 Series
                                Candidates:  1. 04:00.0 ATI Radeon HD 5900 Series
                            That middle entry sure is odd it has it as master and a slave at the same for a possible chain. I think this is a result that between the the fglrx module and the way the kernel is looking at things as it can't make
                            a 3 tier chain internally like Windows can.

                            In short between the fglrx module, the way kernel is looking at things, The MSI-X assignments, and the wonderful code of the proprietary drivers themselves it is in a state of FUBAR when it comes to CrossFire with more
                            than 2 GPU's at least for now.

                            As far as the IOMMU there is more than one implementation of it. The one that hdemo is disabling is the AGP aperture portion which by default reserves 64MB of RAM. With the AMD 890-990FX chipsets which basically are the same exact thing with the exception I have a bit more control with the 990FX IOMMU in BIOS and kernel side.
                            Code:
                            dmesg | grep IOMMU
                            [    0.000000] Please enable the IOMMU option in the BIOS setup
                            [    5.564370] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
                            I enabled IOMMU for AMD-v which is designed for virtual setups more specifically with new features like GPU Pass-through etc.. at the same time I did not allow it to create the 64MB aperture which is a seperate option on my North Bridge for IOMMU.
                            In light of this and now at least how it's all making sense t o me now lol. I will most likely go back to KMS as I have more control and I can use all the features of y IOMMU including the V2 introduced in the 3.3.x Kernels. IMMOU gives you more support for PCI PRI and PASID.
                            Mind you this is markd "experimental". I have feeling this coupled with another new feature introduced in the 3.3.x kernels called DMA_SHARED_BUFFER a.k.a DMA-BUF will open the door for multi-gpu setups using open source drivers. (Which is already being implmented and worked on with Nvidia open source)

                            Comment


                            • #15
                              Originally posted by Acidphase View Post
                              Meaning still "disabled" rather than enabled your in the same boat as me lol.
                              Maybe.

                              Originally posted by bridgman View Post
                              - dump out amdconfig options to see if there are any --initial parms specific to crossfire/dual graphics
                              What you mean under "dump out"? "amdiconfig --adapter=all --delete-crossfire-chain"?

                              Originally posted by bridgman View Post
                              - stop X
                              - run amdconfig --initial adapters=all from console, along with any specific-to-crossfire stuff I found in the amdconfig output
                              - check to see if crossfire candidate is found
                              - if so enable crossfire
                              - start X
                              If I doesn't disable discrete GPU before step three then I get this:
                              Code:
                              amdconfig --list-crossfire-candidates
                              
                              Master adapter:  0. 00:01.0 AMD Radeon HD 6620G
                                  Candidates:  none
                              Master adapter:  1. 01:00.0 AMD Radeon 6600M and 6700M Series
                                  Candidates:  none
                              If I disable discrete GPU before step three then I get this: http://phoronix.com/forums/showthrea...803#post248803
                              So I still at the same point.

                              Originally posted by bridgman View Post
                              I'll ask around and see if there is a better guide, or maybe something in the release/install notes.
                              It's will be much better if CrossFire settings will be available in CCC. Do you agree?

                              Comment

                              Working...
                              X