Revision history
Status/Ver	Date	Drafter(s)	Description of changes	
Draft/1.02	24-Jun-2010	Andrew Raiche	Added Revision history	
Draft/1.03	07-Oct-2010	Andrew Raiche	Added Example with listModules and hwShow; Table of Contents	
Draft/1.04	16-May-2011	Andrew Raiche	Added check FIDs; Reset Panel from commandline.	
Draft/1.05	16-Dec-11	Andrew Raiche	Added BIOS settings, boot switch 7 in VxWorks.	
Draft/1.06	24-Jan-14	Andrew Raiche	Added some Linux Commands, answered some questions.	
				
Status: Draft or Approved - Date: dd-Mmm-yy
Reviewers
Name	Role	
		
Contents
Questions and Answers
Video Frame Boot Switch
IMAGE STORE DIAGNOSTICS
PANEL DIAGNOSTICS
Example: Console commands
Example: Fader Calibration output
Example:  listModules and hwShow on a 2M/E panel
Example: Hardware Module Read outs.
Example: Real Panel Diags
showPanelState : Not sure about application.
Example:  setStateValue (No real world application.)
Gettemp.sh temperature of the panel.
Reset Panel Application from Command line.

Questions and Answers

1.	PCU Menu Proc communication & diagnostics.
A:Windows; Recovery procedure from Stan. 

2.	PCU Panel Proc Linux recovery.
A: Ping; Telnet; U32

3.	PCU Panel Proc communications and diagnostics.
See Below.

4.	PCU Proc data dumps & logs.
SW DIAG example.

5.	PCU proc to stripe / module communication, status and diagnostics.
See Below

6.	PCU proc to module FPGA diagnostics, status and upgrade procedures. 
Menu in Window

7.	Image store communications and diagnostics.
Ping; Telnet.

8.	Image store Linux recovery.

9.	Is my Image store broken or is it the Kontron? How do I know? (Reset is NOT the miracle cure all!)
Kontron does not run the Image Store.  Kontron runs the Video Frame.

10.	Frame Data dumps & logs.
MTVG examples.

11.	What do all of those LEDs going on and off actually mean in a Kayenne? How about in an XL?
Reset; Ethernet; Sync; Serial; FPGA Load; Running iDPMs.

12.	What secret handshakes are there on the panel that would help us?
Calibrate Lever Arms
Stripe Assignment (Different from Port Assignment.)

Video Frame Boot Switch 

>h
Current boot switch setting:
  1 2 3 4 5 6 7 8
 | | | | | | | |
 | | | | | | | +- S8(off) - boot swup.exe or main.exe, see S1
 | | | | | | +--- S7(off) - normal boot messages
 | | | | | +----- S6(off) - boot into NO WAIT HOST mode (i.e. boot normally)
 | | | | +------- S5(off) - do not activate crash dump
 | | | +--------- S4(off) - boot into non-diagnostics mode
 | | +----------- S3(off) - S3 unused
 | +------------- S2(off) - boot using normal IP address information
 +--------------- S1(off) - boot main.exe, if S8 is off
  [1] Status              [2] CPU
  [3] ME                  [4] SS
  [5] Debug               [6] Software Options
  [7] Fan Board           [P] Power Supply
  [H] Boot Sw Help        [D] EnableAllUpdates
  [x] Exit

Boot Switch On explanation.

Switch	Function	
All off	Normal boot of normal application (main.exe)	
1	Boot SWUP (swup.exe)	
2	Boot using factory default IP address information	
3	Boot from alternate directory	
4	Boot into diagnostics mode	
5	Activate Crash Dump (when all other switches are off)	
6	Boot into debug mode (WAIT HOST mode active)	
7	Generate verbose boot process messages	
8	Boot VPITEST (vpitest.exe)	

In v4.0 VxWorks boot switch 7 was updated to allow for serial data input from the console port.  The verbose boot message option was deprecated.

Serial Port Baud 115200 8-N-1 for VxWorks serial console – Kayenne
Serial Port Baud 9600 8-N-1 for KayakHD XL serial console - KayakHD

Menu Information

Q:  What ports does the Menu run on?

IMAGE STORE DIAGNOSTICS

Telnet
Login 
U: root
P: root
cd /opt/missouri/bin  
./mosseth --mgt

Or
/opt/missouri/bin/mosseth --mgt

This command entered in the console will clear up a particular failure on the channels. It will reset the channels.
http://192.168.0.171/cgi-bin/thumbnail.cgi?suite=0001&id=0001&field=0001 

Grab a thumbnail directly using http in a web browser.
ls
ls -l
ps aux

PANEL PCU BIOS Settings

From WP-3016 at N:\Test\Released_Directory\Kayenne\PCU:

6)	Power System and verify BIOS	Connect the VGA and keyboard cables to the Panel Processor at J11 and J7, respectively.
Power on unit. 
As it boots up enter the BIOS setup by hitting the Delete key.  
Verify BIOS version is GV03R015. Write on checklist.  
In main screen set the System Date and System Time.

Verify the following BIOS setting.
In Advanced screen, select the IDE Configuration menu.
Verify Legacy IDE Channels is set to PATA Only.
Press F10 > OK.
Check Fan Air – Physically feel the air from all 3 fans and the verify air flow direction. Check Fan board LEDs are lit.
Check Power LEDs are ON for all the boards.	

17)	Verify BIOS version	Reset the Menu Processor.
As it boots up enter the BIOS setup by hitting the Delete key.  
Verify BIOS version is GV03R015. Write on checklist.

In main screen set the System Date and System Time.
Press F10 > OK.
Boot into Windows.	

PANEL DIAGNOSTICS
 
/home/target/nvram/

/home/target/Data

Panel\target\nvram\PANEL
MISWorkplaces.xml

// Connecting to panel
// Telnet and FTP – 
// User: root  Pass: root 

#screen –r
#screen –dr
 
// takes you to the VP> console that has panel application commands.
VP>

// How to detach a console without exiting the panel application.
[Ctrl-A] d

Linux Console Commands

These are general commands for use at the Linux Console that has the ‘#’ prompt.

// Change directory to root level ‘/’
#cd /

// List directory
#ls

// List directory long format  (Doesn’t work in all Linux use ‘ls -l’)
#ll
#ls –l

// Disk Usage –human readable
#du –h

// Remove <file name>
#rm <file name>

Example: Console commands

VP>help
----------------------------------------------------------------------------
Command list: (type "help <command>" for more information)

bpdel              bpdelall           bplist             bpset
changePDL          continue           dbchk              dbt
dcSet              exit               faderCalib         fpgaStatus
go                 gsSet              halt               help
hwShow             hwevopt            hwevq              ledCalib
linux              listModules        loadModule         macroShow
midSet             pcuchk             pcut               pvmCompact
pvmReset           pvmShow            pvmTest            reboot
reloadModule       resetModule        restart            screensaver
sendFID            setAttr            setLogLevel        setMaxEventTime
setScreensaver     setStateValue      showEth            showEvents
showLogs           showPanelState     showPmem           showSubscribers
showTypes          showVariables      step               stop
traceOff           traceOn            triggerEvent       ucppDisconnect
ucppShow           unsetStateValue    vp
----------------------------------------------------------------------------

Example: Fader Calibration output

VP>faderCalib
command format: faderCalib <workplace id> <group id> <module id> show|reset
VP>faderCalib 1 2 11 show
Fader calibration for widget 1.2.11.410:
min:          4059
max:          4919
inverted:    false
zero xing:   true
old pos:         0
current pos:  4029
EEPROM offs: 604
last raw:     4029
last filtrd:  4029
accu:            0
last dir:    negative
Fader calibration for widget 1.2.11.510:
min:           240
max:          1092
inverted:    true
zero xing:   false
old pos:         0
current pos:  1089
EEPROM offs: 600
last raw:     1089
last filtrd:  1090
accu:           -1
last dir:    positive

Example:  listModules and hwShow on a 2M/E panel

VP>listModules
Config Base Directory: /flash2/config

WPID  GID   MID   MHL file name                  PDL file name
================================================================================
 1     1     1    MHL/RC4316-SourceSelection25.mhl PDL/RC4316-SourceSelection25/MIS.pdl
 1     1    11    MHL/RC4330-Transition_X1.mhl   PDL/RC4330-Transition/MIS.pdl
 1     1    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 1     2     1    MHL/RC4316-SourceSelection25.mhl PDL/RC4316-SourceSelection25/MIS.pdl
 1     2    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 1     2    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 1    10     1    MHL/RC4320-AuxBusControl25.mhl PDL/RC4320-AuxBusControl25/MIS.pdl
 1    11     1    MHL/RC4340-Multifunction.mhl   PDL/RC4340-Multifunction/default.pdl
 1    12     1    MHL/RC4350-MasterEmem.mhl      PDL/RC4350-MasterEmem/MIS.pdl
 1    13     1    MHL/RC4355-MachineControl.mhl  PDL/RC4355-MachineControl/MIS.pdl
 1    14     2    MHL/RC0000-SystembarLong.mhl   PDL/RC0000-SystembarLong/MIS.pdl
VP>
VP>hwShow
Active Modules:

Module ID  	Port  	Conn  	Mod ID	Log ID  Errors (Mod / DB)
1.1.1            	1     	1     	4316    	4312    		0/0
1.1.11    	 1     	3     	4330    	4332    		0/0
1.1.21		 1     	2     	4335    	4337    		0/0
1.2.1      	2     	1     	4316    	4312    		0/0
1.2.11     	2     	2     	4330    	4332    		0/0
1.2.21     	2     	3     	4335    	4337    		0/0
1.10.1     	3     	1     	4320    	4322    		0/0
1.11.1     	1     	4     	4340    	4342    		0/0
1.12.1     	2    	4     	4350    	4352    		0/0
1.13.1     	2     	5     	4355    	4357    		0/0
1.14.1     	1     	7     	4379    	4379    		0/0
1.14.2     	2     	7     	4379    	4379    		0/0
1.14.3     	3    	7     	4379    	4379    		0/0

// Note that the module 4379 is the Distribution board itself but it also reports as the system bar on port 7. Port 7 is an internal connection for the system bar.

command format: hwShow <wp id> <grp id> <mod id> [<section name> [<mux index>]]
VP>

WPID  GID   MID   MHL file name                  PDL file name
================================================================================
 1     1     1    MHL/RC4323-SourceSelection35.mhl PDL/RC4323-SourceSelection35/MIS.pdl
 1     1    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 1     1    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 1     2     1    MHL/RC4323-SourceSelection35.mhl PDL/RC4323-SourceSelection35/MIS.pdl
 1     2    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 1     2    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 1     3     1    MHL/RC4323-SourceSelection35.mhl PDL/RC4323-SourceSelection35/MIS.pdl
 1     3    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 1     3    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 1     4     1    MHL/RC4323-SourceSelection35.mhl PDL/RC4323-SourceSelection35/MIS.pdl
 1     4    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 1     4    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 1    10     1    MHL/RC4326-AuxBusControl35.mhl PDL/RC4326-AuxBusControl35/MIS.pdl
 1    11     1    MHL/RC4340-Multifunction.mhl   PDL/RC4340-Multifunction/default.pdl
 1    12     1    MHL/RC4350-MasterEmem.mhl      PDL/RC4350-MasterEmem/MIS.pdl
 1    13     1    MHL/RC4355-MachineControl.mhl  PDL/RC4355-MachineControl/MIS.pdl
 1    14     4    MHL/RC0000-SystembarLong.mhl   PDL/RC0000-SystembarLong/MIS.pdl
 2     1     1    MHL/RC4316-SourceSelection25.mhl PDL/RC4316-SourceSelection25/MIS.pdl
 2     1    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 2     1    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 2     2     1    MHL/RC4316-SourceSelection25.mhl PDL/RC4316-SourceSelection25/MIS.pdl
 2     2    11    MHL/RC4330-Transition.mhl      PDL/RC4330-Transition/MIS.pdl
 2     2    21    MHL/RC4335-LocalEMEM.mhl       PDL/RC4335-LocalEMEM/MIS.pdl
 2    10     1    MHL/RC4320-AuxBusControl25.mhl PDL/RC4320-AuxBusControl25/MIS.pdl
 2    11     1    MHL/RC4340-Multifunction.mhl   PDL/RC4340-Multifunction/default.pdl
 2    12     1    MHL/RC4350-MasterEmem.mhl      PDL/RC4350-MasterEmem/MIS.pdl
2    14     1    MHL/RC0000-SystembarLong.mhl   PDL/RC0000-SystembarLong/MIS.pdl

Example: Hardware Module Read outs.

VP> listModules
VP>hwShow...

VP>hwShow 1 1 1
Module 1.1.1:
-----------------------------
Connection:       port 1 conn 1
Module Number:    RC4323
TOCOM Number:     85552910.0002
FPGA DS Number:   1281
FPGA Version:     35
Date:             2008-11-3
FPGA Temperature: 55
Ambient Temp.:    52
GX Packet Count:  TX 4645 RX 13445 CRC Errors 0
Pckt Cnt Dst Brd: TX 13445 RX 1342 CRC Errors 0
DMA enabled:      yes
OLED Mux enabled: yes
Sections:         board(U) temp(U) button(U) led(U) led_cfg oled_select(U) oled oled_cfg text text_cfg
VP>hwShow 1 2 1
Module 1.2.1:
-----------------------------
Connection:       port 2 conn 1
Module Number:    RC4323
TOCOM Number:     85552910.0002
FPGA DS Number:   1281
FPGA Version:     35
Date:             2008-11-3
FPGA Temperature: 58
Ambient Temp.:    54
GX Packet Count:  TX 5570 RX 13631 CRC Errors 0
Pckt Cnt Dst Brd: TX 11282 RX 9532 CRC Errors 10
DMA enabled:      yes
OLED Mux enabled: yes
Sections:         board(U) temp(U) button(U) led(U) led_cfg oled_select(U) oled oled_cfg text text_cfg
VP>

Module 1.3.1:
-----------------------------
Connection:       port 3 conn 1
Module Number:    RC4316
TOCOM Number:     00008555.2620
FPGA DS Number:   1281
FPGA Version:     35
Date:             2008-11-3
FPGA Temperature: 59
Ambient Temp.:    53
GX Packet Count:  TX 15986 RX 13363 CRC Errors 0
Pckt Cnt Dst Brd: TX 13363 RX 3133 CRC Errors 10
DMA enabled:      yes
OLED Mux enabled: yes
Sections:         board(U) temp(U) button(U) led(U) led_cfg oled_select(U) oled oled_cfg text text_cfg

Module 1.4.1:
-----------------------------
Connection:       port 4 conn 1
Module Number:    RC4323
TOCOM Number:     85552910.0002
FPGA DS Number:   1281
FPGA Version:     35
Date:             2008-11-3
FPGA Temperature: 56
Ambient Temp.:    52
GX Packet Count:  TX 2066 RX 12845 CRC Errors 0
Pckt Cnt Dst Brd: TX 12845 RX 2055 CRC Errors 0
DMA enabled:      yes
OLED Mux enabled: yes
Sections:         board(U) temp(U) button(U) led(U) led_cfg oled_select(U) oled oled_cfg text text_cfg
---

Example: Real Panel Diags

VP>showPanelState PANEL.DIAG

Panel State:
Function ID                               Value               [Subscribers/Used]
==============================================================================
PANEL.DIAGNOSIS.INFO.PCU.FAN1.SPEED     = "2640"              [ U]
PANEL.DIAGNOSIS.INFO.PCU.FAN2.SPEED     = "2700"              [ U]
PANEL.DIAGNOSIS.INFO.PCU.FAN3.SPEED     = "2760"              [ U]
PANEL.DIAGNOSIS.INFO.PCU.POWER1.PRESENT = "TRUE"              [ U]
PANEL.DIAGNOSIS.INFO.PCU.POWER2.PRESENT = "TRUE"              [ U]
PANEL.DIAGNOSIS.INFO.PCU.TEMPERATURE    = "31"                [ U]
PANEL.DIAGNOSIS.STATUS.PCU.FAN1.SLOW    = "FALSE"             [ U]
PANEL.DIAGNOSIS.STATUS.PCU.FAN2.SLOW    = "FALSE"             [ U]
PANEL.DIAGNOSIS.STATUS.PCU.FAN3.SLOW    = "FALSE"             [ U]
PANEL.DIAGNOSIS.STATUS.PCU.POWER1.FAN.FAULT= "FALSE"             [ U]
PANEL.DIAGNOSIS.STATUS.PCU.POWER1.OFF   = "FALSE"             [ U]
PANEL.DIAGNOSIS.STATUS.PCU.POWER2.FAN.FAULT= "TRUE"              [ U]
PANEL.DIAGNOSIS.STATUS.PCU.POWER2.OFF   = "TRUE"              [ U]
PANEL.DIAGNOSIS.STATUS.PCU.TEMPERATURE.FAULT= "FALSE"             [ U]
==============================================================================
Statistics:
Number of entries:            14
Unsubscribed values:          14       (100%)
Unused values:                0        (0%)
Memory usage:                 1 KB

Total:
Number of entries:            159301
Unsubscribed values:          156493   (98%)
Unused values:                1116     (0%)
Memory usage:                 16036 KB
==============================================================================

showPanelState : Not sure about application.

VP> listModules
-> there should be a number of Transition modules, each of them having a WPID of 1, a GID in the range 1..8 and a MID of 11.

For each listed transition module, enter the following (example: GID=2):

VP>showPanelState PANEL.MODULE.1.2.11.LOGICBOARD.ID

There should be a state entry for each module, with a value of "4332", and flags [SU].

VP>showPanelState MISS1.ME0

Example:  setStateValue (No real world application.)

I've added a new SID

MF.AUX[1-60].TRANSITION.RATE ( 0-999)

Once this SID is defined and a AUX transition resources are available (MF.AUX[1-60].TRANSITION.AVAILABLE -> TRUE), the display show the transition rate value. To check the new implmentation just type the following commands into the VP shell:

VP>setStateValue MISS.AUX1.TRANSITION.AVAILABLE FALSE
VP>setStateValue MISS.AUX1.TRANSITION.AVAILABLE TRUE
VP>setStateValue MISS.AUX1.TRANSITION.RATE 10

Gettemp.sh temperature of the panel.

For temperature on the panel you can use the gettemp.sh script to get the temp.  It displays an average of the FPGA temperature and ambient module temperature.  This script runs on Karrera panels but does not show the correct temperatures.  Andrew is looking into updating the script so it will work on Karrera.  Update 2014-01-24  This has been not approved for support in Karrera.

Ftp "gettemp.sh" to your panel.

Telnet to the panel and at the linux '#' prompt onetime run Dos to Unix to convert the file.
#dos2unix gettemp.sh
#chmod +x gettemp.sh

// For the average of the whole panel it is dot space dot slash gettemp.sh:

#. ./gettemp.sh
PLX device already initialized!
ambient min: 46°C max: 63°C avg: 53°C
fpga    min: 45°C max: 68°C avg: 56°C

// Or for a list of all the modules and the average:

#. ./gettemp -d
PLX device already initialized!
port 1 conn 1: ambient 55°C fpga 62°C
port 1 conn 2: ambient 63°C fpga 68°C
port 1 conn 3: ambient 55°C fpga 63°C
port 1 conn 4: ambient 51°C fpga 57°C
port 1 conn 7: ambient 55°C fpga 46°C
port 2 conn 1: ambient 52°C fpga 60°C
port 2 conn 2: ambient 51°C fpga 56°C
port 2 conn 3: ambient 60°C fpga 61°C
port 2 conn 4: ambient 51°C fpga 58°C
port 2 conn 5: ambient 46°C fpga 49°C
port 2 conn 7: ambient 53°C fpga 45°C
port 3 conn 1: ambient 54°C fpga 60°C
port 3 conn 7: ambient 55°C fpga 46°C
ambient min: 46°C max: 63°C avg: 53°C
fpga    min: 45°C max: 68°C avg: 56°C

// This is what happens when you run it and have not done Dostounix see above.
# . ./gettemp.sh
: not foundemp.sh: line 8:
: not foundemp.sh: line 17:
: not foundemp.sh: line 19: plxInit
: not foundemp.sh: line 20:
-sh: ./gettemp.sh: line 23: syntax error: word unexpected (expecting "do")

Reset Panel Application from Command line.

Telnet to the panel and login 
User: root
Password: root

At the command prompt # enter this command

#killall screen ; KayenneKL.sh

This will stop the application and restart it.

It then takes you to the VP> console on reboot and you should see any boot errors or it should boot correctly.   
VP>

[Ctrl-A] d
You must do a Control A and then press ‘d’ to detach from the VP> console before exiting the telnet session.  If you don’t the panel will You could also leave the telnet session open.

When you have this command prompt 
# 
It is ok to exit the telnet session.

Enable Logging FIDs (No real world application, yet?)

To enable logging of FIDs, the following steps have to be taken:

1.	Telnet into the control panel
2.	Jump to the panel software’s terminal session (screen -dr)
3.	Enter this command: setLogLevel MDM debug1

This system uses WDTs logging system, and there are only two ways to review the FIDs:

1.	Keeping the telnet session open
2.	Looking at the screenLog.0 file in the /home/target/ directory

BIOS settings Kayenne Panel 

From WP-3016 at N:\Test\Released_Directory\Kayenne\PCU:

7)	Power System and verify BIOS	Connect the VGA and keyboard cables to the Panel Processor at J11 and J7, respectively.
Power on unit. 
As it boots up enter the BIOS setup by hitting the Delete key.  
Verify BIOS version is GV03R015. Write on checklist.  
In main screen set the System Date and System Time.

Verify the following BIOS setting.
In Advanced screen, select the IDE Configuration menu.
Verify Legacy IDE Channels is set to PATA Only.
Press F10 > OK.
Check Fan Air – Physically feel the air from all 3 fans and the verify air flow direction. Check Fan board LEDs are lit.
Check Power LEDs are ON for all the boards.	

18)	Verify BIOS version	Reset the Menu Processor.
As it boots up enter the BIOS setup by hitting the Delete key.  
Verify BIOS version is GV03R015. Write on checklist.

In main screen set the System Date and System Time.
Press F10 > OK.
Boot into Windows.