Print

Print


Hi Taylor, 

The ABCDtraining_test folder contains the following files: 10 .Rlog containing the message I specified below as well as 10 numbered .txt files describing the noise components (e.g. [32, 35, 95, 96, 108]). My hand_label_noise files contain an empty line below the line specifying noise components.

The participant’s logMatlab files contain the information below. 

Thanks for your help!

Sandra

                            < M A T L A B (R) >
                  Copyright 1984-2014 The MathWorks, Inc.
                    R2014a (8.3.0.532) 64-bit (glnxa64)
                             February 11, 2014

 
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
 
/scratch.global/sthijssen/shared/DataAnalysis/MRIdata/ABCDpreprocessedData/Subje
cts/sub-NDARINVYFMUX7R3/ses-baselineYear1Arm1/func/feat_analyses/preAroma/sub-ND
ARINVYFMUX7R3_ses-baselineYear1Arm1_run-03_preprocessing_preAROMA_7Nov18.feat/
   114     1

Error Time: 06/27/2019 08:14:54
Index exceeds matrix dimensions.



On Jun 28, 2019, at 10:52, Taylor Hanayik <[log in to unmask]> wrote:

Hi Sandra, 

It seems that you have an “NA” where the name of the csv file should be:

'/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test/NA': No such file or directory

This could indicate that R wasn’t able to locate that file, and the code has placed an NA there and continued on trying to process. 

Could you check your directories and make sure that you do indeed have all the .csv. and .txt files listed? The script is trying to look in:

/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test

Can you list the contents of that folder and send it back?


Cheers,

Taylor Hanayik


Taylor Hanayik PhD
Analysis Research Software Engineer
FMRIB, John Radcliffe Hospital
University of Oxford
[log in to unmask]





On 20 Jun 2019, at 18:33, Sandra Thijssen <[log in to unmask]> wrote:

Dear experts,

A few weeks ago I tried to get your advise on an error I run into when running FIX. I have been using FIX on the melodic.ica output created with AROMA (renamed to filtered_func_data.ica). Is that output by any chance different from what FIX is expecting and could that be what is causing FIX to quit?

I have been receiving the Rlog files described below when trying to create a train-weights file.

Thanks!

Sandra

R version 3.3.3 (2017-03-06) -- "Another Canoe”
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

library(party)
Loading required package: grid
Loading required package: mvtnorm
Loading required package: modeltools
Loading required package: stats4Loading required package: strucchange
Loading required package: zoo

Attaching package: ‘zoo’
The following objects are masked from ‘package:base’:
 as.Date, as.Date.numeric

Loading required package: sandwic
library(e1071
library(kernlab)

Attaching package: ‘kernlab’

The following object is masked from ‘package:modeltools’:

 prior

library(class
library(ROCR)
Loading required package: gplots

Attaching package: ‘gplots’

The following object is masked from ‘package:stats’:

lowess

library(randomForest)
randomForest 4.6-14
Type rfNews() to see new features/changes/bug fixes

library(MASS)

# R CMD BATCH "--no-restore --args ${FIXDIR}/ ${MELOUT}/ Standard 0.1" ${FIXDIR}/fix_2_PREDICT.R ${MELOUT}/fix/logR_Standard_0.1.tx

if (T) {
+ args <- commandArgs(TRUE)
+ path.to.fix <- args[[1]]
+ print(path.to.fix)
+ path.to.train.files <- args[[2]]
+ print(path.to.train.files)
+ weight.file.name <- args[[3]]
+ print(weight.file.name)
+ #
+ #which.fix <- args[[4]]
+ #print(which.fix)
+ } else {
+ path.to.fix <- "/Users/reza/Documents/Academic/FIX/fix1.02”
+ path.to.train.files <- "/Users/reza/Documents/Academic/FIX/raw_data/EXAMPLEL/TRAIN”
+ weight.file.name <- "/Users/reza/Documents/Academic/FIX/raw_data/EXAMPLEL/TRAIN.RData”
+
[1] "/home/lucianam/sthijsse/fix1.067”
[1] "/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test”
[1] "/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test.RData
if (substr(path.to.fix, nchar(path.to.fix), nchar(path.to.fix))!="/“)
+   path.to.fix <- paste(path.to.fix, "/", sep = “")
if (substr(path.to.train.files, nchar(path.to.train.files), nchar(path.to.train.files))!="/“)
+   path.to.train.files <- paste(path.to.train.files, "/", sep = “”

readMelView <- function(file.path) {
+    tmp <- readLines(file.path, -1) # read the full text file
+    tmp <- tmp[grepl("^\\[", tmp) & grepl("\\]$", tmp)] # get THE line
+    noise.list <- as.numeric(strsplit(gsub("\\]", "", gsub("\\[", "", gsub(" ", "", tmp))), ',')[[1]]) # only keep the comma-separated noise IDs > split them
+    return(noise.list) # return a vector, containing noise IDs
+ }

#readMelView <- function(file.path) {
# numb <- runif(1,0,100000000000000)*10
# system(paste("cat ", file.path, " | grep '^\\[' > ", file.path, numb, sep="”))
# con <- file(paste(file.path, numb, sep=""), "rt”)
# tmp <- readLines(con, 1) # Read one line
# close(con)
# noise.list <- as.numeric(strsplit(gsub("\\]", "", gsub("\\[", "", gsub(" ", "", tmp))), ',')[[1]])
# system(paste("rm ", file.path, numb, sep="”))
# return(noise.list)
#}

# pre-processing (e.g., concatenate all files, etc.)
is.consistent <- T
file.feats <- list.files(path.to.train.files, "*csv”)
file.label <- list.files(path.to.train.files, "*txt”)
for(i.feat in file.feats)
+ if(sum(substr(i.feat, 1, nchar(i.feat)-3)==substr(file.label, 1, nchar(file.feats)-3))==0)
+ is.consistent <- F

if (!is.consistent) {
+ print("There is a problem in that the required files are not arranged properly”)
+ print("In order for this function to work, there needs to be two files per MELODIC folder”)
+ print("         1) a features file                  | CSV extracted by FIX”)
+ print("         2) a MELVIEW-compatible label file  | TXT from manual or FIX labeling”)
+ } else {
+ for(i in c(1:length(file.feats))) {
+ tmpf <- read.csv(paste(path.to.train.files, file.feats[i], sep = ""), header = F)
+ tmpl <- readMelView(paste(path.to.train.files, substr(file.feats[i], 1, nchar(file.feats[i])-3), "txt", sep = "”))
+ tmpf$labs <- 1
+ tmpf$labs[tmpl] <- 0
+ if(i==1) train.mat <- tmpf else train.mat <- rbind(train.mat, tmpf)
+ }
+ }
Error in file(file, "rt") : cannot open the connection
Calls: read.csv -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
cannot open file '/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test/NA': No such file or directory
Execution halted

On Jun 4, 2019, at 13:45, Sandra Thijssen <[log in to unmask]> wrote:

Dear experts,

Does anyone have an idea what is going wrong when I try to create a train-weights file using FIX? My Rlog files shows the error below.

Thanks for your help and apologies for reposting.

Kind regards,
Sandra



R version 3.3.3 (2017-03-06) -- "Another Canoe"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.


library(party)
Loading required package: grid
Loading required package: mvtnorm
Loading required package: modeltools
Loading required package: stats4
Loading required package: strucchange
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

 as.Date, as.Date.numeric

Loading required package: sandwich
library(e1071)
library(kernlab)

Attaching package: ‘kernlab’

The following object is masked from ‘package:modeltools’:

 prior

library(class)
library(ROCR)
Loading required package: gplots

Attaching package: ‘gplots’

The following object is masked from ‘package:stats’:

 lowess

library(randomForest)
randomForest 4.6-14
Type rfNews() to see new features/changes/bug fixes.
library(MASS)


# R CMD BATCH "--no-restore --args ${FIXDIR}/ ${MELOUT}/ Standard 0.1" ${FIXDIR}/fix_2_PREDICT.R ${MELOUT}/fix/logR_Standard_0.1.txt

if (T) {
+ args <- commandArgs(TRUE)
+ path.to.fix <- args[[1]]
+ print(path.to.fix)
+ path.to.train.files <- args[[2]]
+ print(path.to.train.files)
+ weight.file.name <- args[[3]]
+ print(weight.file.name)
+ #
+ #which.fix <- args[[4]]
+ #print(which.fix)
+ } else {
+ path.to.fix <- "/Users/reza/Documents/Academic/FIX/fix1.02"
+ path.to.train.files <- "/Users/reza/Documents/Academic/FIX/raw_data/EXAMPLEL/TRAIN"
+ weight.file.name <- "/Users/reza/Documents/Academic/FIX/raw_data/EXAMPLEL/TRAIN.RData"
+ }
[1] "/home/lucianam/sthijsse/fix1.067"
[1] "/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test"
[1] "/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test.RData"

if (substr(path.to.fix, nchar(path.to.fix), nchar(path.to.fix))!="/")
+   path.to.fix <- paste(path.to.fix, "/", sep = "")
if (substr(path.to.train.files, nchar(path.to.train.files), nchar(path.to.train.files))!="/")
+   path.to.train.files <- paste(path.to.train.files, "/", sep = "")

readMelView <- function(file.path) {
+    tmp <- readLines(file.path, -1) # read the full text file
+    tmp <- tmp[grepl("^\\[", tmp) & grepl("\\]$", tmp)] # get THE line
+    noise.list <- as.numeric(strsplit(gsub("\\]", "", gsub("\\[", "", gsub(" ", "", tmp))), ',')[[1]]) # only keep the comma-separated noise IDs > split them
+    return(noise.list) # return a vector, containing noise IDs
+ }

#readMelView <- function(file.path) {
# numb <- runif(1,0,100000000000000)*10
# system(paste("cat ", file.path, " | grep '^\\[' > ", file.path, numb, sep=""))
# con <- file(paste(file.path, numb, sep=""), "rt")
# tmp <- readLines(con, 1) # Read one line
# close(con)
# noise.list <- as.numeric(strsplit(gsub("\\]", "", gsub("\\[", "", gsub(" ", "", tmp))), ',')[[1]])
# system(paste("rm ", file.path, numb, sep=""))
# return(noise.list)
#}

# pre-processing (e.g., concatenate all files, etc.)
is.consistent <- T
file.feats <- list.files(path.to.train.files, "*csv")
file.label <- list.files(path.to.train.files, "*txt")
for(i.feat in file.feats)
+ if(sum(substr(i.feat, 1, nchar(i.feat)-3)==substr(file.label, 1, nchar(file.feats)-3))==0)
+ is.consistent <- F

if (!is.consistent) {
+ print("There is a problem in that the required files are not arranged properly")
+ print("In order for this function to work, there needs to be two files per MELODIC folder")
+ print("         1) a features file                  | CSV extracted by FIX")
+ print("         2) a MELVIEW-compatible label file  | TXT from manual or FIX labeling")
+ } else {
+ for(i in c(1:length(file.feats))) {
+ tmpf <- read.csv(paste(path.to.train.files, file.feats[i], sep = ""), header = F)
+ tmpl <- readMelView(paste(path.to.train.files, substr(file.feats[i], 1, nchar(file.feats[i])-3), "txt", sep = ""))
+ tmpf$labs <- 1
+ tmpf$labs[tmpl] <- 0
+ if(i==1) train.mat <- tmpf else train.mat <- rbind(train.mat, tmpf)
+ }
+ }
Error in file(file, "rt") : cannot open the connection
Calls: read.csv -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
cannot open file '/home/lucianam/shared/DataAnalysis/MRIdata/ABCDrfMRIscripts/ABCDscripts_minimallyProcessed/ABCD_FIXtraining/ABCDtraining_test/NA': No such file or directory
Execution halted

########################################################################

To unsubscribe from the FSL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1


########################################################################

To unsubscribe from the FSL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1



To unsubscribe from the FSL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1




To unsubscribe from the FSL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1