Skip to Content

Category: General News

CTF 17 (England and Wales) and CME files (England only)

Author: Jim Haywood (Capita ESS)

You can ensure that you are using the latest resources that we have provided to help with this issue by referencing the CTF 17/18 Import Contact Issue Key Documents Hot Topic.

Added 26/09/2018 18:00 – SQL Script and SIMS SQL Database Diagnostic Tool to identify last CTF import/export dates and which users have acknowledged the CTF splash screen

The following SQL database script can be run against a SIMS database to identify which users have ticked the CTF incident informational splash screen (please see the below screen print).

The SQL script should be executed at the database management system (DBMS) layer as it is not a part of the SIMS application.

The output will identify the last CTF import and export dates as well as which user has acknowledged the CTF splash screen. Please see the below example.

 

Alternatively the Diagnostic_tool_and_Diagnostic_Script can be employed to generate the same output. Please follow these instructions when using the tool.

 

Added 20/09/2018 at 09:10 – Run (or re-run) Database Diagnostics after applying the Workstation Patch

It is important to run Tools | System Diagnostics | Database Diagnostics  after applying the Workstation Patch as indicated in Step 2 (of the 4 step process) described in the CTF Issue document.  This is true even if Database Diagnostics has been run previously.  Check 99 is improved by the Workstation Patch and is likely to report a slightly wider range of possible issues to check and improved information to dealing with the possible issues.  You will find that much of the advice that help desks are able to offer will be contingent on Database Diagnostics having been run after application of the Workstation Patch.

Added 17 September 2018 at 16:19 – CTF Import of ULNs

We understand that CTF files for Pupil ULNs are now available from S2S in the correct 2018 format.  Using CTF Import to import CTF files for Pupil ULNs has no significance to the CTF Import contact issue, except that school will need to have indicated that they have complete dealing with the CTF Import contact issue before import of CTF files is available from SIMS.

Updated 11/09/2018 at 16:30 – Detailed step by step presentation of the main aspects of dealing with the contact data issue (Version 5.0)

This PowerPoint presentation is available from our Row 03 of the CTF Key Documents Hot Topic.

Important Note

This presentation illustrates how Excel can be used to view a CSV file created from the output for check 99 in Database Diagnostics. This is made available via the application of the workstation patch for either the SIMS Spring Release 2018 or the SIMS Summer Release 2018.

We ask that all schools makes use of this latest check 99 to ensure that all issues created by the incident are identified so that they can be addressed.

Added 07/09/2018 at 17:30 – New Data Collection Sheet (Contacts Only)

A restricted version of the Data Collection sheet for student contacts only has been created in response to customer requests. This Data Collection sheet will populate with basic student details and then provide blank spaces for parental/emergency contact information. This report is helpful if schools wish to provide a simple form for parents to complete to gather or check emergency contact information.

The report needs to be imported into SIMS via Reports | Import – Data Collection (Contacts Only).rptdef. Once imported, you can access the report via Reports | Run Reports | Student to produce for multiple students, or via the Reports link within the student record for an individual pupil.

For Capita Hosted SIMS schools the report will be imported for you by COP Tuesday 11th September.

This report definition file is available from the CTF 17/18 Import Contact Issue Key Documents Hot Topic.

Added 06/09/2018 at 13:05 – Reminder: Informing local authorities and third party partners

Where schools provide information to local authorities, schools should inform their local authority once their SIMS data has been corrected (or inform their local authority that their school(s) was not impacted).  It is also advised that schools inform any third-party partners which may use their SIMS data.

Added 05/09/2018 at 15:00 – Administrative access to Options Online

Please note that administrative access is now available for options online where verification against the CTF incident is still outstanding.  This provides suitable access to carry out the administrative tasks that are necessary at this time of the year.  Full access is resumed once your school indicates that the contact data involved with CTF Import incident has been checked and (where appropriate) remedial action taken.

Added 05/09/2018 at 13:20 – CTF Key Document Hot Topic

We have ‘pinned’ a CTF Key Document Hot Topic to the top of the Hot Topic panel in My Account to help schools with the CTF 17/18 contact incident.  This will stay in place while schools are dealing with this incident and we will ensure that it provides links to the most current versions of the key documents.

Updated 04/09/2018 at 15:40 – Answers to questions posed on the forum (Version 4.0)

Click here to download a PDF document that answers questioned posed on the CTF 17/18 Import Contact Issue forum thread.  This is our fourth version of this document and can be taken to be our view on the 04/09/2018.  We intend to update the document as necessary in the future.  All answers will make it clear as to what date we had the view expressed.

Added 04/09/2018 at 09:35 – ULN and URN are not part of the CTF Import Contact Issue

On the 1st August the CTF version moved on by one year as normal. This means that CTF Export will provide CTF 18 format files, rather than CTF 17 format files, and CTF Import will accept CTF 17 and 18 format files, rather than CTF 16 and 17 format files. The main impact of introducing the CTF 18 format files is that the school’s URN (Unique Reference Number) is mandatory, in addition to the school’s Establishment Number (LA number + school number).

We understand that the CTF files provided via S2S for adding the ULN (Unique Learner Number) for learners is still in CTF 16 format. This format is no longer supported for CTF Import. The DfE will be dealing with this matter shortly, but for the moment we must caution against making manual changes to the file names and content of CTF files to ‘trick’ CTF Import into accepting them, as the formats for CTF 16 and CTF 18 are not compatible.

Please see the normal ‘SIMS Newsfeed – CTF 18’ for any further updates in this area.

Updated 30/08/2018 at 14:10 – Presentation of using MS Excel to view the Check 99 report  (Version 4.0)

This PowerPoint presentation is available from a ZIP Click the Link  CTF_Import_-_Using_Excel_to_view_the_revised_Check_99_report_pptx.

Important Note

This presentation illustrates how Excel can be used to view a CSV file created from the output for check 99 in Database Diagnostics. This is made available via the application of the workstation patch for either the SIMS Spring Release 2018 or the SIMS Summer Release 2018 (intended release on 17th August 2018). Application of either of these patches provide improvements to the original check 99.

We ask that all schools makes use of this latest check 99 to ensure that all issues created by the incident are identified so that they can be addressed.

Version 3.1

Filenames changed so the that they are the same a those used in the latest ‘How to ..’ document.

Scenarios added to the end of the presentation in line with the latest ‘How to ..’ document.

Version 4.0

Section added at the start to describe the mechanics of copying output from Check 99 to a CSV file ready for opening as a spreadsheet.

 

Added 29/08/2018 16:05 – ‘How to Identify …’ document for Capita hosted SIMS 7 customers

Please note that a variant of our ‘ How to Identify and Correct Data Impacted by the CTF Import Incident’ document, for use by Capita hosted SIMS 7 customers, is available from here.

 

Updated 23/08/2018 11:50 – Screenshots after application of the workstation patch

*** The version number is shown in the footer for each page of the document. ***

A PDF document with screenshots captured from SIMS after the workstation patch has been applied is now Resource ID 6503.

Version 2.0: Filename changed to CTF_-_Screenshots.pdf and version number added to footer.

Version 3.0: Further explanation added.

 

Added 21/08/2018 at 18:25 – Answers to questions posed on the forum

CTF Forum QA

A document that answers questioned posed on the CTF 17/18 Import Contact Issue forum thread.  This is our first version of this document and can be taken to be our view on the 21/08/2018.  We intend to update the document as necessary in the future.  All answers will make it clear as to what date we had the view expressed.

Added 17/08/2018 at 13:30 – Improvements to the contact data check in Database Diagnostics (Check 99)

We have released a new workstation patch today following a recent investigation to ensure all potential defect scenarios are captured.

It is important the new workstation patch released today on Friday 17th August 2018 is deployed per school so that both the new diagnostics and corrected CTF logic is in place. Depending on the current SIMS release version the following workstation patch must be applied:

Spring 2018 workstation patch 1 – CTF Import & Check 99 Improvements.
Summer 2018 workstation patch 1 – CTF Import & Check 99 Improvements.

Capita hosted SIMS customers are already on the SIMS 2018 Summer Release (7.182) and therefore the “Summer 2018 workstation patch 1” will be applied on Friday 17th August 2018 outside of working hours.

Does the workstation patch need to be applied?

If your school does not use the CTF process the school will not be impacted (this can be checked in SIMS CTF log, from Routines | CTF Import | View History Log). The school will not need to take any further action.

In all other instances the workstation patch should be applied.

Next steps

Once the correct workstation patch has been applied the new diagnostics within the patch must be run and remedial action taken. Please READ HERE for a detailed description this.

Please note: Even if you have previously cleaned the incorrect contact merge data (based on advice received in July), this patch will identify potentially more incorrect contact information which needs to be addressed (i.e. contacts not at the same address).

Access to the new diagnostic script for Support Units

We have purposefully not provided access to the new diagnostic script by default to Support Units. This is to ensure the new workstation patch (above) has been applied first. Once the new patch has been applied you can request access to the script from Capita SIMS Service Desk.

Existing Modules

Several SIMS modules access and use contact information from SIMS. We therefore have taken the proactive precautionary step for impacted SIMS modules as previously published (see also a summary below).

SIMS Product Product Functionality Suspended Product Functionality Still Available
Online Services
SIMS Parent Calendar | Access to Child’s Dashboard

(attendance, behaviour &

achievement, assessment, timetable,

reports, homework and the data

collection) | Message generation |

Homework

School Detail | School term dates
SIMS Parent Lite Access to Child’s Data Collection School Detail | School term dates
SIMS Activities All areas No functionality, i.e. switched off
SIMS Student All areas No functionality, i.e. switched off
SIMS Options All areas No functionality, i.e. switched off
Other modules
SIMS Agora All areas No functionality, i.e. switched off

 

From Monday 20th August schools will be able to directly reactivate SIMS Online Services and Agora from their respective administration sites and therefore easily reinstate product functionality. Schools will be asked first whether they have completed the data clean-up and will need to acknowledge they are happy to reactivate the service.

Schools should only use InTouch (SIMS text an email module) for generic communications to parents and contacts. Any communication which contains specific pupil information should not be sent until the schools are confident they have validated their student contact data.

Splash screen

New splash screen notifications with acceptance buttons has been included in the SIMS7 workstation patch to ensure all schools are aware of the incident and have taken the necessary action to address potentially incorrect data.

The splash screen references this document for background information and guidance on how to undertake the necessary remedial action.

Additional questions

If you have any questions about the content of this communication or would like additional detail, please contact the Capita SIMS Service Desk.

Added 17/08/2018 at 12:45 – Presentation of using MS Excel to view the Check 99 report

This PowerPoint presentation is available from a ZIP  RESOURCEID-1-6464.

Important Note

This presentation illustrates how Excel can be used to view a CSV file created from the output for check 99 in Database Diagnostics. This is made available via the application of the workstation patch for either the SIMS Spring Release 2018 or the SIMS Summer Release 2018 (intended release on 17th August 2018). Application of either of these patches provide improvements to the original check 99.

We ask that all schools makes use of this latest check 99 to ensure that all issues created by the incident are identified so that they can be addressed.

Version 3.1

Filenames changed so the that they are the same a those used in the latest ‘How to ..’ document.

Scenarios added to the end of the presentation in line with the latest ‘How to ..’ document.

Updated 15/08/2018 at 19:45 – Improvements to the contact data check in Database Diagnostics (Check 99)

The improved version of Database Diagnostic Check 99 includes:

  1. A more focused search eliminating some of the near duplicate lines (i.e. less to focus upon).
  2. Improved formatting and simpler column headings.
  3. Further identification of the applicant/pupils details that are linked to contacts (i.e. inclusion of unique pupil number).
  4. The list of checks displayed at the end of the diagnostic process will now display “Check 99” in black text were no issues have been found and in red text where one or more issues have been found. This brings Check 99 into line with the other diagnostic checks.

NEW FEATURE: The diagnostic check now includes contacts where the applicant/pupil has a different address to that of the contact (such as social workers/divorced parents).

When our consolidated patches are available, schools should apply patch 24203 where the Spring Release 2018 is in use, or patch 24212 where the Summer Release 2018 is in use.

NB: We will be taking a slightly different approach to delivering the changes outlined above and it is now our intention to release on Friday 17th August a new patch and associated guide for the SIMS Spring and Summer releases. This is subject to the satisfactory completion of our testing and UAT processes.

 

Added 14/08/2018 at 14:30 – CTF matching incident – Precautionary measures for SIMS Modules using contact data.

Several SIMS modules access and use contact information from SIMS which may have been impacted by the CTF incident.  We have therefore taken the proactive precautionary step for SIMS modules as below:

SIMS Product Product Functionality Suspended Product Functionality Still Available
Online Services
SIMS Parent Calendar | Access to Child’s Dashboard

(attendance, behaviour &

achievement, assessment, timetable,

reports, homework and the data

collection) | Message generation |

Homework

School Detail | School term dates
SIMS Parent Lite Access to Child’s Data Collection School Detail | School term dates
SIMS Activities All areas No functionality, i.e. switched off
SIMS Student All areas No functionality, i.e. switched off
SIMS Options All areas No functionality, i.e. switched off
Other modules
SIMS Agora All areas No functionality, i.e. switched off

 

Consolidated patches

It is our intention to release on Friday 17th August a new patch and associated guide for the SIMS Spring and Summer releases. This is subject to the satisfactory completion of our testing and UAT processes. This patch will need to be applied by all customers. Capita hosted SIMS customers will be updated out of hours on the day of release.

This patch will update the diagnostics tool (covering incorrect links for contacts who did not live at the same address as the pupil, for example social worker/divorced parents) and ensure the CTF import is working as expected.
This new patch must be applied, diagnostics undertaken, and manual fixes completed before the SIMS Modules can be reactivated.

Reactivation of SIMS modules

Following the release of the consolidated patch schools will be able to reactivate directly their SIMS Online Services listed above through the Product Admin Site (https://admin.sims.co.uk). Schools will be asked first whether they have completed the data clean-up and will need to acknowledge they are happy to reactivate their service.
Similarly, schools will be able to reactivate Agora through its admin site (https://www.simsagora.co.uk).

Advice on messaging

Schools should only use InTouch (SIMS text and email module) for generic communications to parents and contacts. Any communication which contains specific pupil information should not be sent until the schools are confident they have validated their pupils’ contact data.

Next steps and additional questions

Further communications will be provided when the patch is available for release or additional guidance need to be given.

Added 07/08/2018 at 08:40 – Improvements to the contact data check in Database Diagnostics (Check 99)

The improved version of Database Diagnostic Check 99 includes:

  1. A more focused search eliminating some of the near duplicate lines (i.e. less to focus upon).
  2. Improved formatting and simpler column headings.
  3. Further identification of the applicant/pupils details that are linked to contacts (i.e. inclusion of unique pupil number).
  4. The list of checks displayed at the end of the diagnostic process will now display “Check 99” in black text were no issues have been found and in red text where one or more issues have been found. This brings Check 99 into line with the other diagnostic checks.

NEW FEATURE: The diagnostic check now includes contacts where the applicant/pupil has a different address to that of the contact (such as social workers/divorced parents).

We expect to be able to make this improved version of Check 99 available at the close of business on Wednesday the 8th August via two consolidated patches. Consolidated patch 24203 should be applied by schools who are using our Spring Release 2018, while consolidated patch 24212 should be applied by schools who are using our Summer Release 2018.

Added 02/08/2018 at 18:10 – This CTF 17 newsfeed will be maintained beyond it normal closure date of 1st  August

As you know, for all schools that have upgraded to the SIMS 7 Summer Release 2018, CTF Export uses the format for CTF 18 instead of CTF 17 from the 1st August.  My normal procedure would be to reflect this in the My Account notification area by making the newsfeed for CTF 17 unavailable.  However, this year I will leave this newsfeed for CTF 17 available so that all items related to our problem with CTF import of contact data is kept available from a single newsfeed.

I have added a link from this newsfeed to the CTF 18 newsfeed and I have also added a link from the CTF 18 newsfeed to this newsfeed.

Updated 23/07/2018 at 17:40 – Reduced scope variant of Database Diagnostic check 99

Check 99 (and its counterpart for DbDiagnose EXE) is very thorough in that it will report on problems with contacts for applicants even when those applicants are also enrolled as pupils at the school. We believe that this approach is in line with quite a strict interpretation of data protection regulations, but we understand that at this stage some schools and support units would appreciate the option of being able to use a variant of check 99 (and its counterpart for DbDiagnose EXE) where issues that relate to applicants that are also enrolments pupils are excluded from the scope of the check.

With the above in mind, we are looking into providing a reduced scope variant of check 99 (and its counterpart for DbDiagnose EXE). If we decide to go ahead, the change for Database Diagnose would be provide via a database patch (which would be made available from SOLUS) and the change for DbDiagnose EXE would be provided via a script (Please contact us or click here).

We expect this ‘variant’ to only be of interest to a minority of schools and, with that in mind, the full scope of check 99 will be adopted for the Autumn Release 2018.

Updated 20/07/2018 at 17:40 – Issues with CTF Import

A detailed FAQ document covering the recent CTF Import issues experienced by some of our customers has been made available from – UPDATED SSUs – How to Identify and Correct Data Impacted by the CTF Import Incident–  this will address your questions and provide clear guidance on next steps.

Thank you for your patience.

Added 18/07/2018 at 11:45 – Clarification regarding overwriting of contact address

Please note that from the SIMS 7 Summer Release 2018 onwards CTF Import cannot replace an existing contact address with an address from the CTF file.  The only circumstances where the contact address in a CTF file will be written to a school’s system is where the contact did not previously exist in the school’s system or where the relevant contact in the school’s system does not already have an address.

Updated 17/07/2018 at 23:25 – Patch 24157 for contact linking and address problem

Please note that patch 24157 has been uploaded to SOLUS and made available to support units for application on systems at the Autumn Release and those upgraded to the Summer Release.  The patch adds check 99 toTools | System Diagnostics | Database Diagnostics to identify definite and likely problems with a school’s contacts and contacts addresses that are likely to have been caused by our contact/address bug.  The reporting of check 99 in Database Diagnostics uses the same basic structure as we have used for the DbDiagnose script and this DbDiagnose script has also been updated (version 4) on CTFContactAddressDiagnoseScriptAndInstructions so that it is line with check 99.

Normally, it is only checks that fail that Database Diagnostics displays in red, but this is not the case for check 99, as this is shown in red even if the check has passed in all cases.  Unfortunately, this is the ‘price we pay’ for the reporting of check 99 being suitable for copying to a spreadsheet.  Where no lines are reported for check 99 failures, we still produce a header line for use with a spreadsheet and Database Diagnostics takes the header line to be an indication of a fail.  We understand that this may cause some confusion, but we believe the approach to be justified in that any school with a significant number of check 99 failures will want to copy the data to a spreadsheet.

A PowerPoint presentation to help with the use of the report from DbDiagnose EXE and with the use of the new check 99 reporting in Database Diagnostics is available from Solus 3

Updated 16/07/2018 at 14:15 – The way forward for the contact linking and address problem

Although patch 24040 has caused some further problems for some of the very small number of schools who applied the patch, it has illustrated the risk involved in attempting to fix data which could have been damaged some months ago and therefore subject to any number of changes since then. With that in mind we now plan to provide as much help as possible for schools to detect the problems, so that they can deal with their data problem using their own knowledge and judgement.

With the above in mind we will be modifying the current script for the DbDiagnose application (which is run outside of SIMS .net) to provide a more focussed view of the contact/address issues that have resulted from our bug in CTF Import to an admission group. Once we are happy with this script, we will convert the logic into a SQL patch to add the check to the SIMS .net route Tools | System Diagnostics | Database Diagnostics. Providing the data check via a SIMS .net route will make the checking more accessible to schools and particularly schools with limited access to direct technical support.

Patch 24157 will add check 99 to Database Diagnostics to identify definite and likely problems with a schools contacts and contacts addresses that are likely to have been caused by our contact/address bug.  This patch is in the final stages of testing and we expect to be able to make it available in the near future.

I would like to apologise for the original bug in our software that triggered the data problem and the amount of time it is taking us to provide an acceptable solution.

Added 11/07/2018 at 15:20 – The way forward for the contact linking and address problem

Although patch 24040 has caused some further problems for some of the very small number of schools who applied the patch, it has illustrated the risk involved in attempting to fix data which could have been damaged some months ago and therefore subject to any number of changes since then. With that in mind we now plan to provide as much help as possible for schools to detect the problems, so that they can deal with their data problem using their own knowledge and judgement.

With the above in mind we will be modifying the current script for the DbDiagnose application (which is run outside of SIMS .net) to provide a more focussed view of the contact/address issues that have resulted from our bug in CTF Import to an admission group. Once we are happy with this script, we will convert the logic into a SQL patch to add the check to the SIMS .net route Tools | System Diagnostics | Database Diagnostics. Providing the data check via a SIMS .net route will make the checking more accessible to schools and particularly schools with limited access to direct technical support.

I would like to apologise for the original bug in our software that triggered the data problem and the amount of time it is taking us to provide an acceptable solution.

Updated 11/07/2018 at 11:45 – Patch 24040 withdrawn

I am very sorry to have to report that patch 24040 has been withdrawn.  A customer case has revealed a situation where the patch can assign incorrect addresses for a contact.  Dealing with this issue is our highest priority.

Our initial investigation of the issue with the patch suggests that the problem is confined to situations where schools have attempted to deal with the contact/address problem ‘manually’ before applying the patch.  For example, the patch may malfunction where a school deleted data and then re-ran the CTF Import before applying the patch.

We understand that application of the patch has made the changes proposed by the diagnose script, so schools that did not agree with the changes proposed by the diagnose script will not have applied the patch.

As you would expect, it is safe to continue to use the contact/address script, but the indication of where a data fixing patch should be applied may not be accurate.

Added 10/07/2018 at 17:20 – Possible problem when downloading CTF files from websites

We understand that some schools are having problems when importing assessment CTF files that were downloaded from a government website.  Our investigations indicate that the problem is caused by the CTF filename being ‘corrupted’ on download.  When a file of the same name already exists in an internet browser’s download folder the browser often adds something to the end of the filename to differentiate the files, e.g. (1) or ~1.  The spurious characters at the end of the CTF filename will need to be removed for a successful import into SIMS.

Updated 10/07/2018 at 16:00 – Contact Linking & Contact Address Problem

Apology

  • As many of you will know, there was a problem with a contact matching in CTF Import to an admission group provided by our SIMS 7 Autumn 2017 and Spring 2018 releases that could result in new pre-admission pupils being linked to the wrong contact and can also result in contacts being allocated incorrect addresses.  Please accept my apology for this mistake in our software.

Description of Problem

  • The SIMS 7 Autumn Release 2017 included a patch for CTF Import that has a serious issue where CTF Import is used for adding pupils to an admission group.

First aspect of the problem

  • On import, the software uses a matching routine to check if any of the pupils to be imported have contacts that already exist in SIMS and, where this is the case, links that contact to the pupil and updates the contacts address as necessary.
  • Unfortunately, several aspects of the matching routine where accidentally removed so that the matching routine only checks for matching names and gender.
  • The net result is that existing contacts can be given the wrong address and the imported pupil in the admission group can be linked to the wrong contacts.

Second aspect of the problem

  • This is similar to the first aspect of the problem, but applies where no address for the contact has been included in the CTF file.
  • As for the first aspect of the problem, the link from the imported pupil will be to the first existing contact with the same name and gender, but in this case the address for that contact is not changed.

Example of Problem

  • Having upgraded to the SIMS 7 Autumn Release 2017 or Spring Release 2018, a school runs Routine | Data In | Import CTF.
  • School chooses ‘Add data for New Pupils Only’.
  • Having selected the CTF file, the user opts for placing the new pupil in a Pre-Admission Group.
  • Pupil named Paul Smith is added to a Pre-Admission and the software looks in the SIMS list of all contacts to find Paul’s contact whose name is John Smith with a gender of male.
  • The software finds the first male John Smith in the list, which given the commonness of the name is likely to be the wrong John Smith, and links that John Smith as a contact for the Paul Smith that has been added to the Pre-Admission Group.
  • Where the CTF file includes an address for the contact, the software compares the address of the John Smith in the CTF file with the John Smith that has been linked to the Paul Smith that has been added to the Pre-Admission Group and, in nearly all cases, finds that they are different.  Having found that they are different the software changes the current address for the contact (which is very likely to be the right address) to the address in the CTF file (which is very likely to be the wrong address).
  • Therefore, the Paul Smith that has been added to the Pre-Admission Group is very likely to have ended up linked to the wrong John Smith.  If this John Smith has been given the wrong address, that wrong address will be significant to both Paul Smith and any other pupils where John Smith is the contact.

Solution

Removing the bug

    1. Patch 24068 has been provided to customers via SOLUS 3 to fix the mistake in our coding (this replaced patch 23971 which only fixed part of the mistake).
    2. Patch 24068 has been included in the SIMS 7 Summer Release 2018 to ensure that the coding fix is in place for all customers.

Dealing with the data problems caused by the bug

    1. A diagnose script has been provided via My Account to help school identify problems in the contact linking and contact address area.  The diagnose script is now at version 3, but all versions are suitable for use when deciding whether or not to apply patch 24040, with the only difference between the three versions being that the later versions are easier to make use of as they are better at excluded data that has already been dealt with manually.
    2. A presentation has been provided via My Account to help with making use of the said diagnose script.  The My Account link now takes you to version 1.1 of the presentation, which includes a description of each type of ‘Potential Issue’.  You can check if you have version 1.1 of the presentation as the version number appears in the top right-hand corner of the first slide.
    3. Patch 24040 has been developed to automatically resolve many of the data problems caused by the bug and is available to local support units via SOLUS.  The patch is described as “DO NOT USE unless you are happy with the address proposed by the CTF Contact Address Diagnose Script” as a small number of schools have disagreed with the changes proposed by the Diagnose Script and such schools should not make use of this patch as the patch will put in places the proposals made by the Diagnose Script.  We believe that the proposals made by the Diagnose Script are correct, but if a school disagrees they should not make use of the patch.  We are sorry that this patch took longer to develop and test than originally expected, but I can assure you that this work has been treated as our number one priority.

Updated 10/07/2018 at 10:50 – Summary of latest position regarding the contact linking and contact address problem

Correcting the bug

  1. Patch 24068 has been provided to customers via SOLUS 3 to fix the mistake in our coding (this replaced patch 23971 which only fixed part of the mistake).
  2. Patch 24068 has been included in the SIMS 7 Summer Release 2018 to ensure that the coding fix is in place for all customers.

Updated 21/06/2018 at 18:30 – Contact Linking & Contact Address Problem Description of Problem

  • The SIMS 7 Autumn Release 2017 included a patch for CTF Import that has a serious issue where CTF Import is used for adding pupils to an admission group.

First aspect of the problem

  • On import, the software uses a matching routine to check if any of the pupils to be imported have contacts that already exist in SIMS and, where this is the case, links that contact to the pupil and updates the contacts address as necessary.
  • Unfortunately, several aspects of the matching routine where accidentally removed so that the matching routine only checks for matching names and gender.
  • The net result is that existing contacts can be given the wrong address and the imported pupil in the admission group can be linked to the wrong contacts.

Second aspect of the problem

  • This is similar to the first aspect of the problem, but applies where no address for the contact has been included in the CTF file.
  • As for the first aspect of the problem, the link from the imported pupil will be to the first existing contact with the same name and gender, but in this case the address for that contact is not changed.

Example of Problem

  • Having upgraded to the SIMS 7 Autumn Release 2017 or Spring Release 2018, a school runs Routine | Data In | Import CTF.
  • School chooses ‘Add data for New Pupils Only’.
  • Having selected the CTF file, the user opts for placing the new pupil in a Pre-Admission Group.
  • Pupil named Paul Smith is added to a Pre-Admission and the software looks in the SIMS list of all contacts to find Paul’s contact whose name is John Smith with a gender of male.
  • The software finds the first male John Smith in the list, which given the commonness of the name is likely to be the wrong John Smith, and links that John Smith as a contact for the Paul Smith that has been added to the Pre-Admission Group.
  • Where the CTF file includes an address for the contact, the software compares the address of the John Smith in the CTF file with the John Smith that has been linked to the Paul Smith that has been added to the Pre-Admission Group and, in nearly all cases, finds that they are different.  Having found that they are different the software changes the current address for the contact (which is very likely to be the right address) to the address in the CTF file (which is very likely to be the wrong address).
  • Therefore, the Paul Smith that has been added to the Pre-Admission Group is very likely to have ended up linked to the wrong John Smith.  If this John Smith has been given the wrong address, that wrong address will be significant to both Paul Smith and any other pupils where John Smith is the contact.

Solution

  • Applying Patch 23971 will fix the first aspect of the bug (available from SOLUS 3).
  • We are creating a second patch that will incorporate Patch 23971, to fix the first aspect of the bug, and will additionally fix the second aspect of the patch.  It will be fine for a school to apply the second patch when they have already applied the first patch and it will also be fine for a school to apply the second patch without having applied the first patch
  • Optionally, make use of a routine, which we expect to be able to provide in the near future, to detect if any of a school’s contact data has been affected by this bug.
  • We expect to be able to make patch 24040 available in the near future to restore the original address for the contacts and to link the imported pupils to the correct contact.  However, it is important to note that this restoration will only be possible where the school has not already attempted to resolve the issue for a contact.

Progress

  • Patch 23971 is available.
  • Patch to replace 23971 is being developed and should be available in the near future
  • Patch 24040 is being extended to take account of both aspects of the issue

Added 09/05/2018 at 14:30 – Problem with importing Proficiency in English (England only)

Unfortunately, there is a problem with our coding of CTF Import which results in any Proficiency in English data not being imported.  The problem is solved by patch 23974, which is being made available from SOLUS.

Added 15/03/2018 at 12:30 – Address Problem (KB478399)

Apology

  • Please accept my apology for not warning you of this problem sooner

Description of Problem

  • Where a pupil is imported into SIMS, when they already have a sibling at the school and the siblings have the same contacts, the addresses for the contacts are ended on the date of import.
  • Where the pupil is set to ‘inherit’ a contact’s address the address for that pupil will also be ended.

Solution

  • Apply patch 23690 to fix the bug
  • Re-import the CTF to restore the addresses
  • This patch will be included in the Summer Release 2018

Added 09/05/2018 at 14:30 – Problem with importing Proficiency in English (England only)

Unfortunately, there is a problem with our coding of CTF Import which results in any Proficiency in English data not being imported.  The problem is solved by patch 23974, which is being made available from SOLUS.

Added 15/03/2018 at 12:30 – Address Problem (KB478399)

Apology

  • Please accept my apology for not warning you of this problem sooner

Description of Problem

  • Where a pupil is imported into SIMS, when they already have a sibling at the school and the siblings have the same contacts, the addresses for the contacts are ended on the date of import.
  • Where the pupil is set to ‘inherit’ a contact’s address the address for that pupil will also be ended.

Solution

  • Apply patch 23690 to fix the bug (available from SOLUS 3)
  • Re-import the CTF to restore the addresses
  • This patch will be included in the Summer Release 2018

Added 04/12/2017 at 16:00 – Restricting export/import of contact data 

Currently we have a tick box that schools can use to indicate whether to export/import contact data.  There has been some discussion of extending the choice to export/import of parental contacts only, e.g.

from

Student Contacts  [  ] 

to

 All Student Contacts  [  ]
 Student Parental Contacts  [  ]

Please raise/support a Change Request on My Account, if you would value such an extension of CTF functionality.

Added 22/11/2017 at 09:35 – Problem with the transfer of service child indicator 

Unfortunately, there is a problem with our coding of CTF Export that can result in the service child indicator being assigned to pupils in the CTF file that are not actually service children.  A patch to resolve this problem will be made available from SOLUS 3 in the near future.

Added 22/09/2017 at 19:10 – Use of effective date for selecting pupils for CME files (England only) 

Unfortunately, in the original SIMS Summer Release 2017 there was a mistake in the coding for selecting pupils for CME files which meant that the System Date was used where the Effective Date should have been used.  Most of you will not have experienced this problem, as Workstation Patch 1 dealt with this problem as one of the CTF pupil selection fixes.  I apologise for not making this clear earlier.

Updated 21st September 2017 at 12:05 – Current academic year restriction for creation of CME files (England only)  

Unfortunately, in the original SIMS Summer Release 2017 there was a mistake in the coding for selecting pupils for CME files which meant that the System Date was used where the Effective Date should have been used.  Most of you will not have experienced this problem, as Workstation Patch 1 dealt with this problem as one of the CTF pupil selection fixes.  I apologise for not making this clear earlier.

Updated 15th August 2017 at 12:25 – Presentation of changes CTF 17 and the introduction of CME files for SIMS 7  

Unfortunately, in the original SIMS Summer Release 2017 there was a mistake in the coding for selecting pupils for CME files which meant that the System Date was used where the Effective Date should have been used.  Most of you will not have experienced this problem, as Workstation Patch 1 dealt with this problem as one of the CTF pupil selection fixes.  I apologise for not making this clear earlier.

A new version of this PowerPoint demonstration was uploaded to My Account on 15/08/2017 at 12:20.

Version 2.1 corrects a mistake regarding the inclusion of Grounds for Removal in CMJ files.

Version 2.0 includes extensive improvements to the presentation, which now includes slides for each of the following areas.

  • SIMS 7 Summer Release 2017
  • SIMS 7 Autumn Release 2017
  • Changes for CTF 17 Compared CTF 16
  • Legal position for CME files
  • CME files for Independent Schools
  • CME files and Grounds for Removal
  • Using CTF, CML and CMJ to avoid CME
  • Using SIMS 7 to enter CME leavers information
  • Using SIMS 7 to create CME Leavers files
  • Structure and Content of CME for Leavers files
  • Trigger for providing CME Joiners information
  • Using SIMS 7 to enter CME joiners information
  • Using SIMS 7 to create CME Joiners files
  • Structure and Content of CME Joiners files

Added 28th July 2017 15:30 – Content of CME (Children Missing Education) Files

Unfortunately, in the original SIMS Summer Release 2017 there was a mistake in the coding for selecting pupils for CME files which meant that the System Date was used where the Effective Date should have been used.  Most of you will not have experienced this problem, as Workstation Patch 1 dealt with this problem as one of the CTF pupil selection fixes.  I apologise for not making this clear earlier.

CML File (Children Missing Education – Leavers)

  • Identification of the pupil
    • UPN
    • Surname
    • Forename
    • DOB
    • Gender
  • Current addresses of the pupil
  • Current phone numbers of the pupil
  • Email address of the pupil
  • Contacts for the pupil
    • For each contact: Relationship to the pupil, Indication of parental responsibility, Identification of the contact, Title, Surname, Forename, Gender, Current address of the contact, Current phone numbers of the contact, Current email address of the contact
  • School the pupil has left
    • Identification of school: School name, School LA, School establishment number
    • Reason for leaving the school
    • Grounds for removal from the admissions register*1
  • School the pupil is expected to join
    • Identification of school: School name, School LA, School establishment number
    • Expected date of entry

 

CMJ File (Children Missing Education – Joiners)

  • Identification of the pupil
    • UPN
    • Surname
    • Forename
    • DOB
    • Gender
  • Current addresses of the pupil
  • Current phone numbers of the pupil
  • Email address of the pupil
  • Contacts for the pupil
    • For each contact: Relationship to the pupil, Indication of parental responsibility, Identification of the contact, Title, Surname, Forename, Gender, Current address of the contact, Current phone numbers of the contact, Current email address of the contact
  • School the pupil has joined
    • Identification of school: School name, School LA, School establishment number
    • Date of entry
  • School the pupil left
    • Identification of school: School name, School LA, School establishment number
    • Date of leaving
    • Reason for leaving the school
    • Grounds for removal from the admissions register*1

*1 Will be introduced by SIMS 7 Autumn Release 2017 in line with the change to the DfE specification for CTF 17.

Product Development for Data Retention

Product Development for Data Retention

Updated with a new video of the bulk student delete process for Autumn

25/09/2018

Notification Updates

25/09/2018: After we packaged the Autumn 2018 release together for the Beta version, we continued further deep integrated testing on the last part of the student delete. This last step is to delete the central Person Record if the student role was the only role that Person had, this is explained in the video below at the point of 3 minutes and 50 seconds. Therefore, for the Autumn 2018 release of SIMS we will still include the last three data domains in the bulk student delete option, but felt it was safer to not deliver in the main customer release the last ‘Delete All Student Data’ option. We raise your awareness to this now as this option is present in the beta but will not be delivered in the release in November. If you need further clarification, please contact paul.featherstone@capita.co.uk.

28/08/2018: We have now completed the development of the bulk student delete routine in SIMS.  This builds on the initial phase which we released in the Summer version of SIMS and now allows for the complete delete of student records.

How will this work?

To help explain the process, I have recorded a video for SIMS customers to review what has been completed.  While the development is complete, we are now working on other GDPR priorities, the team would welcome feedback on the updates.  So please review the video and then provide your comments in the feedback form below.

The Video…

Additional Information

Please feel free to share this video with anyone who will benefit from knowing what new functionality will be available in the Autumn release.  The direct link is: https://youtu.be/w8sp3XFviIM

The developers are now working of some GDPR features to do with PLinks between SIMS and FMS and then will start to relax some of the rules prohibiting users to delete members of staff from SIMS.

Feedback

Knowing what you think about this routine is important, even if it is to know we are doing it right.  Please use the form below to submit your comments.

SIMS 7 – CTF Issue

SIMS 7 – CTF Issue

Background information

What is CTF?

CTF stands for “Common Transfer File”. This is the mechanism used for loading new pupil applicants and transfers from other schools into SIMS.

What is the issue?

Incorrect matching of applicants and on roll students imported by CTF into your SIMS7 database may have linked new pupil records to an existing contact. Additionally, some data on the existing contact record may have been changed.

Will this issue have impacted my school(s)?

Not all schools will be impacted.

  • If your school does not use the CTF process you will not be impacted (this can be checked in SIMS CTF log, from Routines | CTF Import | View History Log). You will not need to take any further action.
  • If your school uses the CTF process you must complete steps 1 to 4 (described below). Your school may still not have been impacted, but you must follow the process outlined nonetheless, to confirm whether you have been impacted and respond accordingly.

Under what circumstance would my school(s) be impacted?

This would happen if a combination of the name, title, gender and relationship of a pupil’s contact record in the CTF matched the same elements of the contact of another applicant, on roll pupil or leaver already held in your database.

One type of example would be:

  • Already on SIMS
  • Pupil = Joanne Bradshaw, with a Contact/Parent = Wendy Bradshaw, Mother, Female (original)
  • New CTF applicant
  • Pupil = Jamie Bradshaw, with a Contact/Parent = Wendy Bradshaw, Mother, Female (new/different Wendy)

In this instance as both pupils have a parent called Wendy Bradshaw, identified also as being Female and with a relationship of Mother, the new CTF applicant would be linked to the original Wendy Bradshaw. The original Wendy Bradshaw address, telephone number and email would also be updated with the new/different Wendy Bradshaw’s details.

Which data may have been altered?

Address, phone number and email address fields on the contact record may be incorrect.

What steps do I need to take?

Please see the following document for a detailed description of the steps that must be taken. How to guide

The following description is a summary of this.

Step 1 – Patch SIMS

It is important the new workstation patch released on Friday 17th August 2018 is run against your school(s) so that both the new diagnostics and corrected CTF logic is in place. Depending on your current SIMS release version the following workstation patch must be applied:

  • Spring 2018 workstation patch 1 – CTF Import & Check 99 Improvements.
  • Summer 2018 workstation patch 1 – CTF Import & Check 99 Improvements.

Capita hosted SIMS customers are already on the SIMS 2018 Summer Release (7.182) and therefore the “Summer 2018 workstation patch 1” will be applied on Friday 17th August 2018 outside of working hours.

Step 2 – Run the diagnostics

As part of the patch (applied in Step 1) an extra check “Check 99” has been added to Database Diagnostics which is run from within the SIMS. “Check 99” will output data identified as possibly incorrect.

Please see the following report for detail on this diagnostic tool. Diagnostic guide

Step 3 – Complete remedial action (if appropriate)

If the diagnostic tool (Check 99) has not identified any actions to take, you can skip this step (Step 3) and complete Step 4.

If potential output data is identified by the diagnostics tool (Check 99) manual checks will need to be made by your school and contact data potentially updated. Please see the following document for details on how to do this. How to guide

Step 4 – Run the clean-up patch

Deploy via Solus the clean-up patch 24213 or contact your usual SIMS support team. This script will clean-up back-end applicant data associated with step 3.

Capita hosted customers can request the deployment of Patch 24213 using this online form. Request Form

NOTE

Any staff member attempting to run a CFT export/import will be challenged in SIMS as to whether they have completed step 1 to 4 described above. It is imperative that your school has completed these steps before any further CTF imports are run.

On login to SIMS a reminder splash screen will draw your (and other administrative and key staff’s) attention to the CTF issue. This reminder screen will be removed only once it has been confirmed on the CTF explore/import page that your school has completed Steps 1 to Step 4.

If I need help implementing patches, running diagnostics or making changes, who can I speak to?

Your usual SIMS support team will be able to help you with these processes.

Exams Days 15th and 22nd August 2018

Arrangements for exam results download days 15th and 22nd August 2018

 

Dear colleague

 

I write with an update on arrangements for the support of exams download days. Please pass this information on to your school’s examinations officer as I may not have their contact details.

 

Results Day Survival Kit

The key to a successful and stress-free download day is preparation. Please read and act upon the advice in the Results Day Survival Kit in advance of download day

 

OSMIS Web Site

We will update the Examinations area of the OSMIS Web Site with the very latest information – please refer back to this site for late breaking information

 

Arrangements for Support on Download Days (15th and 22nd August)

The OSMIS Service Desk will be open from 7am on download days. As the volume of calls is difficult to predict (depends on factors outside our control), we ask that you log calls for support via email wherever possible (support@osmis.co.uk) and provide a direct dial number on which we can call you back on. Logging calls in this way will mean that we are able to prioritise calls appropriately and are able to provide a solution at first contact in more cases. Of course 01618831290 is still there but we ask for your support in ensuring that we manage the load and deliver the best service possible at this busy time.

 

Do I Need to Apply the SIMS Summer Release?

If the school is not making use of PI functionality in SIMS (using SISRA, Datasec or similar), then the SIMS Summer Release is not required to download results to the SIMS system.

 

Are There Any Database Patches I Need to Apply?

This all depends on your intended use of SIMS but at the time of writing the following patches have been released by Capita:

  • Fileset 1002 – Required if the school has previously imported Fileset 1001
    • The earlier fileset did not include the latest Examindirection.txt file which lists the files to be used for the PI detail reports this year. Please click here to download fileset 1002 which will need to be unzipped and then imported via Tools>Setups>Import Fileset
  • 24126 – Correction to how single Science qualifications count in Attainment 8
    • This patch corrects an error when calculating the Attainment 8 figure for a student who has only taken two single sciences.
  • 24015 – Update to Assessment Manager Exams Gradeset JC A/2
    • This patch updates the Grade Points in the Assessment Manager Gradeset for JC A/2 to:
      • A Grade 25
      • B Grade 20
      • C Grade 15
      • D Grade 10
      • E Grade 5
      • U Unclassified 0
      • X No result 0
      • Q Pending 0
  • 23876 – Update to Assessment Manager Exams Gradeset JC 41
    • This patch updates the Assessment Manager Exams Gradeset JC 41 to the correct values (this gradeset is commonly used for double Science awards).
  • 23539 – Update to JC Gradesets for GCSE 9-1 Short Courses
    • Applying patch 23539 will update Exams JC Gradesets JC 40/2 Result and JC 44/2 Result and enable the import of 9-1 short course basedata.

And Finally

If you have any concerns regarding the download of Exam Results, then please don’t wait – log a call and we’ll be in touch to assist.

 

With best wishes for a smooth and successful download day

Summer Holidays

Dear All,

With the start of the summer holidays the Connect-Up help desk will now be operating at a reduced level, this is due to staff members taking holiday and also major work projects.

We hope you understand and wish you a peaceful and happy summer.

Regards,

Connect-Up Support Team.

Third Party Software

Dear Customers,

We are receiving a an ever increasing number of calls, on the help desk regarding third party software products (i.e. Classroom Monitor, Medical Tracker, Call Parents, Parent Pay, Inventry etc.) and we are rapidly approaching the point where it will affect our ability to fulfil our core responsibility to school of supporting SIMS and their statutory responsibilities.

In light of this we thought it would be best to clarify the position of Connect-Up, regarding the support of third party products.

Connect-Up has the responsibility, under the SIMS SLA, to assist 3rd party suppliers apply patches, facilitate remote access for investigation and/or assisting with network or SIMS configurations required prior to installation.

We would advise any customers purchasing 3rd products to ensure they have also purchased installation from the supplier. Our recommendation for the next step with this is to contact the supplier of the software and ask them when they can install this for you.

They may need a remote connection to the server/computer that this needs setting up on, if that is the case can you please put them in touch with us as we are more than happy to facilitate a remote connection for them to fulfil the installation.

Please bear in mind that we will need an Email confirmation from a senior member of staff at the school pre-authorising us to give them access to your server/computer to make the relevant changes, this is to cover both yourselves and us under GDPR.

If the school has not got installation included in the purchase of any 3rd party software, an option for you as a school would be that if you do have a managed service contract with us, you are more than welcome to use their time as you please whilst they are in your school (that would need to be agreed with senior staff within the school and the engineer made aware before their site visit).

As a last option if installation has not been purchased and the customer is unable to setup the 3rd party product purchased then Connect-Up would have to charge for a dedicated engineer to install the product.

Key Stage Assessment Returns

Important information for Key Stage Returns

Instructions for 2018 Key Stage Assessment Returns via Capita SIMS .

Connect-Up Support Services GDPR Preperations.

The upcoming changes to the data privacy law are very extensive and daunting to schools.  Connect-Up has continued its approach of producing informed briefings, toolkit focused training courses and hands-on practical workshops, in order to provide the skills your school will need to meet these new complicated and legal requirements from May 2018.

Part of the process that all schools should be doing is to check that companies who process data on your behalf are making the same preparation efforts for the new law that you in school are making.  The majority of companies are simply adding details to their websites (Connect-up are doing this as well) but this requires you to actively search for these.  Connect-Up is trying to make to process simpler by contacting you directly with our standard processor letter which explains the steps we take when processing any from you in a legal and secure manner.  Please click the link below for a copy of our letter.

Connect-Up Processor Letter

Don’t forget that the General Data Protection Regulation comes into force on the 25th of May and that we offer a wide range of training and consultancy on data protection to our schools including

All Staff Briefings

SLT Briefings

Data Lead / Chief Privacy Officer Training

Gatekeepers Training

Progress Check / GAP Analysis Consultancy

If you would like to request further information or discuss your school’s data compliance, please feel free to contact me.

Kindest regards
Peter Brookes
School Compliance Advisor/Statutory Consultant
Connect-Up Support Services

CES Census 2018 (Catholic Schools in England and Wales)

CES Census 2018 (Catholic Schools in England and Wales)

 

Updated 22/01/2018 at 12:50 – Issues with reporting of religions other than Catholic

Unfortunately, there is a problem with the way in which we are counting religions other than Catholic that can result in underreporting of such religions.  For example, if a school decided to create additional religion Lookup Codes for Sunni and Shi’ite Muslims and set the Categories for the Lookup Codes to Muslim, only the pupils original Muslim Lookup Code would be included as Muslim and the addition Lookups Codes would be counted as no religion.

We apologise for this mistake on our part and have made patch 23611 available from SOLUS to correct his mistake.

Once the patch has been applied, schools should proceed as follows.

1. Open Tools | Lookups | Maintain and select Home Religion as the lookup type.

2. Ensure that each Active religion code has an appropriate religion category, e.g. a code for Church of England should have a category of CH (Christian).

3. Run Create & Validate for the Census return to ensure that the correct category data is reported in the Census.

We have added steps 1 and 2 above to this newsfeed item, as we have found that the religion mapping is missing in some schools, which has resulted in the religion being reported as missing in the return for those schools.

 

Added 03/01/2018 at 09:50 – Issues with reporting of religions other than Catholic

Unfortunately, there is a problem with the way in which we are counting religions other than Catholic that can result in underreporting of such religions. For example, if a school decided to create additional religion Lookup Codes for Sunni and Shi’ite Muslims and set the Categories for the Lookup Codes to Muslim, only the pupils original Muslim Lookup Code would be included as Muslim and the addition Lookups Codes would be counted as no religion.

A patch 23611 available and will be depolyed to all Catholic Schools.

Fileset 803 required for Spring Census 2018

Capita has released another Fileset_803 to address changes made by the DfE in the upcoming census on the 18th January 2018 

We will be  deploying this Fileset to our supported customers but please ensure you have Fileset_803 listed in the statutory returns area of SIMS. (Routines | Statutory Returns | School Census)

If you do not have Fileset_803 listed please contact  the helpdesk on 0113 2105008